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ABSTRACT 

In a 3-year study of the impact of education 
consultation en development of continuing medical education (CME) 
programs in ccmir.unity hospitals, results seem to indicate that: (1) 
limited inputs by physicians, acting as education consultants, can 
significantly facilitate development of hcspital-based CME prograirs. 
Over a 2-year period, 40 hospitals, comprising a stratified sample of 
acute, short-term community general hospitals in four New Encland 
states, received education consultation and training of theii 
physician education coordinators. Following consultation, trc.ined 
behavioral scientists visited both the ftO consulted hospital/j and a 
matched group of 40 control hospitals that did not receive 
consultation. Consultation was shown to be effective on both scores: 
consulted hospitals implemented 60% of the consultants' 
recommendations within 18 months after they were submitted; and a 
statistically significant greater number of educational changes 
occurred at consulted hospitals than at the matched group of control 
hospitals . (Author/HS) 
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ABSTRACT 



CONTRACT NIH 70-4150 



IMPACT OF EDUCATION CONSULTATION ON DEVELOPMENT OF 
CONTINUING MEDICAL EDUCATION PROGRAMS IN COMMUNITY HOSPITALS 



In a three year study of the impact of education con- 
sultation on development of continuing medical education 
(CME) programs in community hospitals, results seem to indi- 
cate that : 1) limited inputs by physicians, acting as educa- 
tion consultants, can significantly facilitate development 
of hospital-based CME programs; and 2) with such consultation 
community hospitals regardless of size or location can develop 
and maintain CME programs. Over a two year period, 40 
hospitals, comprising a stratified sample of acute, short 
term, community general hospitals in four New England states 
received education consultation and training of their 
physician education coordinators. Consultants made 
several visits each year to their hospitals and collaborated 
with key hospital personnel, including a locally dei,ignated 
physician education coordinator, to 1) review ongoing 
education programs, 2) identify educational needs and 
local resources, 3) structure and initiate new pro- 
grams, 4) provide access to extra-hospital resources. Consultants 
submitted sets of formal written recommendations to each 
of their hospitals, tailored to the local situations 
During tha second year of the study, both consultants and 
loca*' coordinators t„ook part in five one-day training 
programs designed to sharpen the consultants' skills 
and facilitate the local coordinators' development as 
hospital medical educators. 

Following consultation, trained behavioral scientists 
visited both the 40 consulted hospitals and a matched grouD 
of 40 control hospitals which did not receive consultation* 
The evaluators collected data on two criteria for effective 
consultation: 1) that hospitals accept the consultants' 
recommendations, and 2) that more education changes occur 
at experimental hospitals than at control hospitals. 

Consultation was shown to be effective on both scores: 
consulted hospitals implemented 60% of the consultants' 
recommendations within 18 months after they were submitted- 
and a statistically significant greater number of education 
changes occurred at consulted hospitals than at the matched 
group of control hospitals. 

Throughout the course of the study, both consultants 
and local coordinators expressed a need for additional 
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guidance. In response, Norman Stearns, MD , Marjorie 
Getchell, MA, and Robert Gold, EdM, collaborated to 
author Continuing Medical Education at Cor.rr.unity Hospitals : 
A Manual for Proi;ram Development , which was published in 
May, 1971 as a supplement to the New England Journal of 
Medicine . Based on a problem-solving application of 
basic scientific methodology called *'A Systematic Approach 
to Developing Education Programs",* the manual presents how- 
to-do-it guidelines for developing hospitil -based continuing 
education programs for physicians^ The m.\nual also contains 
guidelines for giving educational consultation to community 
hospitals . 

Two major projects undertaken by PMI were a direct 
outgrowth of the present contract: the use of education 
consultation to stimulate physician e ducat ion programs 
addressed to the unique problems of the poor in 
hospitals located in depressed areas, and the development 
of core libraries in community hospitals. 
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BACKGROUND 



CONTINUING MEDICAL EDUCATION AT COMMUNITY HOSPITALS 
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Rapid advances in the development of medical theory 
and practice during the past two decades has had a dual 
impact on today's practicing physician. First, medical 
ca?e progress has resulted in a continuously accumulating 
body of knowledge which the physician must ;.eep abreast 
of if he is to provide optimal care to his patients; 
and second, it has magnified the relationship of the 
practicing physician to his community hospital. 

For the competent physician to practice medicine 
effectively, he must have at his disposal a vast and ever- 
developing' body of information pertinent to the care of 
his patients. It has been widely held that the best 
■edical school education can be obsolete within five years 
of graduation, creating a vacuum which the jr .ysician must 
fill by continuously furthering his own education. The 
increasing need for continuing medical education is 
clearly presented in the November 1967 report of the 
National Advisory Commission on Health Manpower which 
called for periodic relicensing of physiciars. The re- 
port reconr/ended relicensing based upon either certifica- 
tion of acceptable performance in continuing education pro- 
grams or upon challenge examinations in the practitioner s 
specialty.* Recent changes in the case law. requiring that 
physicians keep abreast of advances in medical science, 
present another compelling reason for concern. In Brune 
?s Belinkoff". the Massachusetts Supreme Court overruled. 
as'being uns uited to present day conditions, the rule which 
measures a physician's conduct by the standards of other 
doctors in his own or similar communities. The new stand- 
ard is that the Physician must exercise the same degree 
of care and skill as other practitioners in the same specialty, 
regardless of where they practice. 



^Report of the National Advisory Commission on Health 
Manpower. Washington : U.S. Government Printing Office, 

'''''4rinrv!-'BninJoff!-3S4 Mass. 102, 235 N.E.2d 793, 
1968. 
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The physician has traditionally relied upon journals 
and periodic professional meetings as primary modes of 
acquiring new medical information. However, he is increasing- 
ly finding that these methods are no longer solely suf- 
ficient to meet his educational needs. With the myriad 
of materials available from these sources, the physician 
feels that he can afford neither the time to isolate rele- 
vant material nor, frequently, to relate their substance 
to hi$ practical patient care problems. 

Even the physician who is strongly motivated to seek 
out alternative means of education finds that responsibilities 
of practice and lack of coherent and accessible programs, 
org an i zed to meet his educational needs, frustrate his 
educational ambitions. Enormous expenditures have been di- 
rected toward accumulat ion of new medical knowledge; lacking 
are corresponding expenditures to develop and provide new, 
more effective methods of disseminating this knowledge. 

In addition to increasing the individual educational 
responsibility of the physician, the advancing technical 
complexity of medical care has also magnified the relation- 
ship between the physician and th- community hospital. 
The physician is more dependent upon the hospital*s ser- 
vices and facilities to provide optimum care for his patients. 
Consequently, his time- spent at the hospital and his in- 
volvement with it and its various activities has increased 
markedly . 

The need for comprehensive and relevant continuing 
education programs , coupled with the physician's close 
relationship with the hospital, provides a rationale for 
the thesis that the community hospital is a logical setting 
for the physician's continuing education. There 'are unique 
advantages to a hospital-based continuing education program 
for physicians. In view of severe limitations of his avail- 
able time, it is practical to plan a program which can take 
place as an integral part of the physician's daily activities, 
and at a location where he regularly attends his patients 
and meets with his colleagues. It is meaningful, also, to 
conduct a program for physicians at a place where education 
can be directly related to the care of his patients. Further, 
it is possible to incorporate a variety of educational for- 
mats and techniques into a hospital-based education program, 
and to take advantage of teaching materials and devices 
not otherwise available to the individual practitioner. 
Lastly, in view of the diversity of practitioner specializa- 
tion and special practice interests, an education program 
that is directed at a small and somewhat constant segment 



of the total physician population, such as a community 
hospital's medical staff, can be based on identified 
learner needs and geared in both content and depth to his 
level of understanding. 

Supportive of this concept of a hospital -based con- 
tinuing education program is legislative history accompany- 
ing Public Law 89-239 (Regional Medical Program) which 
points out that within the community hospital setting, 
continuing medical education programs can be designed to 
have real and immediate relevance to problems faced by 
the physician in his daily practices. 



PMI'S APPROACH TO HOSPITAL-BASED CONTINUING MEDICAL EDUCATION 

For the past seventeen years, Postgraduate Medical In- 
stitute (PMI) has focused its efforts on developing and 
implementing continuing medical education programs for the 
physician at his place of daily p:'actice -- the community 
hospital . 

PMI was established in 1953 as a non-profit organiza- 
tion under the sponsorship of the Massachusetts Medical 
Society. Its objectives were stated as follows: 

It will coordinate activities of all agencies 
concerned with postgraduate education and in 
particular it will call in the expert help of 
the medical schools of the Commonwealth. It 
will conduct courses carefully planned for the 
needs of practicing physicians but it will in 
no way i-iterfere with or compete with the 
teaching activities of established groups that 
wish to carry on their own programs. It is 
prepared to offer assistance to community 
hospitals or similar groups who wish to devel- 
op local programs. Finally, it will be in a 
position to experiment with and find various 
new teaching techniques in this most difficult 
of fields.^ 



^Laurence B. Ellis. Reflections on Postgraduate Medi- 
cal Education for Practicing Physicians. New England Jour - 
nal of Medicine, 250: 243-246, 1954. 



The Institute began its activities by developing and 
implementing a lecture series for hospitals and district 
medical societies throughout Massachusetts. It was soon 
realized, however, that the isolated lecture was insuf- 
ficient to fulfill the physician's educational needs, 
and that the development of coordinated and continuous 
hospital-based education programs • would be a more appropri- 
ate undertaking. 

PMI urged that the following principles, based upon 
adult education theory, be incorporated into the develop- 
ment of the education program: 

1. Include the physician in the planning and 
implementation of the education program; 

2. Relate the program to the physician's edu- 
cational needs and interests; 

3. Define specific educational program objec- 
tives which correspond to identified needs 
and interests of the physician; and 

4. Utilize teaching methods and materials 
appropriate to the program content. 

PMI activated these principles in the form of a 
consultation program, whereby visiting medical education 
consultants assisted individual community hospitals, 
regardless of size, to improve their current education 
programs or to undertake new ones. 

The consultation process was primarily a "catalytic 
stimulant" to the hospital. As an energizing force, it 
helped others address, analyze and solve their educational 
problems. Hospitals and their medical staffs were en- 
couraged to develop collectively their own coordinated 
programs of continuing education based upon the above 
principles. 

Tho methods of consultation consisted of several key 
steps in this self-help process. Although it was well 
recognized that continuing physician education leads to 
improved patient care, PMI realized that the prospect of 
improved patient care was often not enough to induce active 
participation on the part of very busy staff members. As 
a result. Step 1 in the development of the hospital -b ased 
continuing education program was to stimulate interest in 
hospital-baseu education and to encourage the commitment of 



key personnel, including medical staff, administration 
and trustees. It was this commitment of individual physicians 
and local hospitals to their own continuing education which 
would most significantly affect patient care. 

Having obtained both commitment and support of hospital 
and staff, the consultant was then able to assist the 
hospital in pinpointing educational resources within the 
hospital that could be used in the development of educational 
programs. Step 2 in the consultation process consisted of 
identifying the educational needs and interests of physician 
staff members as a basis for defining objectives of the 
e ducat i on program . 

Step 3 consisted of identifying resources in terms of 
finances , educational materials and facilities, and personnel. 
Emphasis was placed on using local strengths for program 
pi anning and impl ementation . In areas where strengths of 
the hospital were not adequate, methods were undertaken 
either to develop them or to supplement them with resources 
of near-by teaching hospitals or medical centers. 

With the proper guidance provided by the consultation, 
the community hospital could develop a viable program on 
its own initiative, *xnd, in some instances, to a significant 
degree, with its own talents. 

An article written by Dr. Norman Stearns, PMI Executive 
Director (See Appendix A), for the New England Journal of 
Medicine , entitled "Positive Approaches to Continuing Medi- 
cal Education in Communi ty Hospitals", describes various 
elements of the consultation process and program development 
in community hospitals. 



INTRODUCTION TO PRESENT STUDY 



Until 1967, PMI had been able to provide consultation 
only on a limited basis, utilizing the services of two 
staff physician consultants . In addition , PMI lacked the 
resources to study systematically the effectiveness of 
consultation. In May of 1967, the Division of Physician 
Manpower, Bureau of Health Professions Education and Man- 
power Training, National Institutes of Health, con- 



tracted^ with PMI to conduct a one year study of consul- 
tation as a means of developing continuing education pro- 
grams at the community hospital. Under the contract, PMI 
would train and utilize additional consultants to deliver 
consultation to a greater number of hospitals. It was 
hoped that the expanded consultation services would result 
in a rapid increase in the number of hospitals with active 
programs of continuing medical educatioa* 

At the completion of the first year of the contract, 
the Division of Physician Manpower felt that the study 
conducted by PMI warranted further implementation and 
documentation; it was also felt that greater emphasis 
should be placed on the training of PMI*s consultants and 
hospital education program coordinators. As a result, 
the contract was extended for another two years. Since 
the original contract was to be of one year's duration, 
the project's design was based on a one year implementation 
period and was carried out as such. The two year extension 
period essentially continued the study of consultation 
services to hospitals with some modifications in design 
as derived from the first year's experiences. 

The present report embodies a presentation and dis- 
cussion of the objectives of the study, the methods used 
in pursuing these objectives, the extent to which .they 
were achi eved , prob lems encountered , and the further use to 
which such methods and findings could be made -applicable. 
Discussion of the final report is divided into two sections: 
PHASE I , whi ch incorporates activities implemented during 
the time period of May 1967 through May 1968, that is, the 
initial contract period; and PHASE II, which covers activities 
occurring from June 19 68 through May 1970 . 



^Contract NIH 70-4150 (PH 108-67-*170) 



PHASE I 



STUDY OBJECTIVES 



The overall purpose of the contract was to study the 
impact of an education consultation that Postgraduate 
Jiedical Institute had provided to community hospitals 
for a number of rears. Consultation was to aid the com- 
^u^ity hospital in determining, organizing and ;n,plement- 
Tn^Us own program of continuing medical education (CME) 
i^led on identified needs and utilizing interna resources 

supplemented by resources of the ""^^^^^J/^^f.^^^ ' .f^^^, 
consultation process was viewed as an attempt to effect 
Change! that is, change in community hospital continuing 
medical education programs. Underlying this premise was 
the a^su^ption tha? cLnges i^^ME programs wou d improve 
the Quality of patient care, the ultimate poal of any 
QIE program. It was not the purpose of this study to 
determine the effect of consultation on the improvement 
of quality of care rendered patients. f ^ J;^,^",!" 
sion of the philosophy and procedures of the consulta- 
Uon appears- in the Introduction to the present report, pp 

The primary objective of the study was to determine 
the effectiveness of the consultation process as a means 
of stimulating and implementing ^^fj^^^^^^edical edu- 
cation program: at community hospitals. Criteria tor 
effec?ire consultation were established: 1) that hospi- 
tals accept the consultants' recommendations, 2) that 
more education changes occur at experimental hospitals 
thll at control hospital during the study period, and 
Sr^hat measurable growth of CME program development 
oLu^ at experimental hospitals. A -o-°J ^ °^ ^ ^J',^"" 
was the determination of factors associated with the 
cons^iltation or the community hospital which tended to 
facilitate or inhibit the impact of consultation. 

J- % m o includes the Scope of Work sub- 

(AppendiX B, pp.Bl-2 , mciuaes tnc ^ f^, . „ ^t,^ rnntract' 
ii??ed by the "contracting agency describing the contract 

objectives and procedures.) 



STUDY DESIGN 



Relevant to the project design, the research question 
posed was of the nature, "What is the effect of X, e.g., 
consultation; cn Y, e.g., development of medical educa- 
tion programs," X being the independent variable and Y 
being the depe-.Jent variable. The dependent variable 
refers to the behavior being measured and was operation- 
ally defined in the present investigation as the nature 
of changes relating to continuing medical education 
occurring in hospitals. The independent variable refers 
to variables which may affect or influence the dependent 
variable. Independent variables included: 

1) the treatment variable, which refers to the 
variable manipulated by the investigator, in 
this case, the consultation process; and 

2) fixed variables, which refer to those over 
which the investigator has no control. They 
pre-exist and are hypothesi::ed to influence 
the dependent variaijle. ror the purposes of 
the present investigation, they were opera- 
tionally defined as characteristics relating 
to the hospital, medical staff or consultant. 

Evaluation of the effectiveness of consultation 
(the primary research objective) focused on: 1) assess- 
ing changes ir. the education program at experimental 
hospitals through examination of the nature and quantity 
of education activities before and after consultation, 
and through assessment of implementation of consultant 
recommendations; and 2) assessing the consequent amount 
of change induced at experimental hospitals relativ., *o 
spontaneously occurring change at a matched group of 
control hospitals. The following evaluation design 
Strategies were formulated: 

Pre-Post Experimental Group Design 

In order to determine changes in experimental hos- 
pitals* education programs, a series of data depicting 
the quantity and nature of CME activities would be 
gathered fror. experimental hospitals prior to and follow- 
ing the medical education consultant visits and would 
undergo comparative analysis. Pre-data would be collected 
by the consultant at the time of his initial visit. 



Comparison of pre-data with post-data, collected by 
behavioral science evaluators fol lowing consultation 
using the sane survey instruments and techniques as the 
Consultants used for gathering pre-data, would denote 
changes in the education prograns accruing to interven- 
tion by consultants. 

Ideall>, the comparison of the results of this 
analysis with that of similar analysis of data gathered 
from a matched group of control hospitals not receiving 
consultation would yield a more valid measurement of 
the impact of consultation. .The difficulty in obtain- 
such data Crom a group of control hospitals plus the 
potential r^ffect that the data collection process (es- 
pecially pre-data collection) would have on the develop- 
ment of CME programs at control hospitals precluded the 
use of a control versus experimental group design in 
this aspect of the study. 

As consultation was seen as a change inducing process 
and the generation and submission of formal written recom- 
mendatio3\s were to serve as ^'ie primary influencing mecha- 
nism, a further critical aspe-t of the pre«post experi- 
mental group design would center on assessing the extent 
to which hospitals implemented recommendations . This 
aspect of the study design dovetailed with the pre-post 
design in the respect that the consultant's recommenda- 
tions were to be based on his appraisal of the hospital's 
areas of needed change at the onset of consul tat ion • 
Hence, assuming a valid appraisal by the consultant, the 
extent to which hospitals implenented recommendations 
would provide a post-measure of hospitals' progress as 
a result of consultation. 



Control vs. Experimental Group Design 

In order to adequately determine whether results 
would be due to consultation rather than to extrr.neous 
or concomitant variables, including the effect of nornal 
growth over the study period, post-consultation change 
data obtained from a control group of hospitals not 
receiving consultation would be compared with comparable 
data gathered from experimental hospitals. Data would 
reflect the nature and extent of hospital -report ed 
changes, both educational and non-educational, occurring 
within each group of hospitals. Change, for study pur- 
poses, was defined as any reported alteration or addition 



to existing hospital organization or operational function , 
including services, personnel and facilities, occurring 
in the interval from the onset of consultation to the 
time of follov-up. Data would be collected by behavioral 
science interviewers after consultation had terminated. 

In keeping with the corollary research objective of 
uncovering factors, both facilitative and inhibitive, 
which might affect the impact of consultation, a series 
of measures on a large number of factors hypothesized to 
be potential influencing factors -vould be gathered for 
both experimcTital (consulted) and control (non-consulted) 
hospitals. Measures would include characteristics re- 
lating to hospital and medical staff attributes within 
both experimental and control hospitals, and consultant 
and consultation attributes within the experimental hos- 
pitals. Project design called for examination of these 
factors as th-ey related to, or affected, the various 
"change measures". (Graphic illustration of the Phase 
I study design is portrayed in Figure 1, p.il3 



PLAN OF PROGRESS 



Procedures established for implementation and eva- 
luation of the consultation process were as follows. 
Project plans called for recruiting and training physician 
academicians who would provide consultation to a selected 
group of community hospitals in the states of Maine, Massa- 
chusetts, New Hampshire, and Rhode Island. Consultants 
would make a minimum of two visits to each of their 
assigned hospitals, meet with key members of the medical 
staff and administration, and discuss various problems of 
and possibilities for the -hospital. • s continuing medical 
education program. At the time of his initial visit 
the consultant would collect information relevant to the 
status of the hospital's education- prograc. The data 
would serve both as a guide for the consultant in deter- 
mining needed program improvements and as a pre measure 
in evaluating the effect of consultation on program 
development. Following the consultation visits, the con- 
sultant would formulate a series of reconizendations per- 
tinent to the needs of the hospital^ submit them to PMI 
for review acd then forward them to the hospital. (A 
detailed discussion of the consultation procedure is 
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contained in the section on consultant training and 
orientation, pp. 23-27) A four month inplementnt ion 
period would follow in which the hospital would have an 
opportunity to act on any or all of the recommcndat ion > . 
Although a longer implementation period would have pro- 
vided a more accurate estimate of consultation impact , 
the time limitations of a one year study precluded any 
such considerations. Following the allotted iraplemen- 
tation period, interviews would be conducted by I a.^i* 
oral scientists with key hospital personnel such as the 
hospital physician education coordinator, administrator, 
president of medical staff, member of the board of trus- 
tees, and the director of nursing, to determine the 
status of change within the hospitals, both in terms of 
reconmendat ions implemented and of other hospital 
changes that may reflect the impact of the consultation. 
In order to v?lidate the effect of the consultation, 
anal.ogous follow-up interviews would be conducted in a 
matched group of non-consulted hospitals. The remain- 
ing period for the one year study would be devoted to 
data analysis and final report writing. 

Achievement of this planned procedure was effected 
in all but a few instances. The following chapter on 
project implementation discusses planned procedures xn 
relation to actual acco.rpl ishment s . (A graphic ?l^n of 
Progress, page 13, illustrates both the planned pro- 
cedures and the actual achievements.) 



PROJECT IMPLEMENTATION 



The main purpose of the present section is to des« 
cribe the overall methodology and execut ion of the study 
Included will be a discussion of how hospitals were 
selected for the study, how individual consultants were 
recruited and trained, how consultation proceeded, and 
how the data were collected, processed and analyzed. In 
general, the section will deal with the progression of 
the project from inception to completion. 

. RECRUITMENT AN1> SELECTION OF HOSPITALS 

The following procedures were employed in recruit- 
ment and selection of a sample of hospitals to receive 
consultation: 
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Planned Schedule 
Actual Achicven?ent 



1. Definition of the population from which the 
sample would be drawn. 

2. Solicitation of the population to determine 
hospitals' willingness to participate in the 
slurfy . 

3. Sijltxtion of a "representative, non-biased" 
sampl e according to s ccified parameters. 

Characteristics of the study population were defined 
as follows: 

1. Type of hospital--short term stay, community 
general hospital.* 

2. Location of hospital-- located in the states of 
either Maine, Massachusetts, New Hampshire or 
Rhode I s land . 

3. Involvement with PMI --hospitals were not cur- 
rently to be involved to any significant degree 
with other PMI activities relating to hospital^ 
based continuing medical education programs . 

One hundred eighty-eight hospitals in the four* New 
England states ^.constituted the study population. 

Accordingly, a letter was mailed to all hospitals 
in the study population introducing PMI, its goals and 
objectives, and inviting them to participate in a study 
to develop, through consultation, continuing education 
programs for physicians at the hospital. Appendix C 
contains a sample copy of the letter of invitation and 
the form on which the hospital could indicate its 
willingness to receive consailtat ion • 

Of the 188 hospitals solicited, 51 initially res- 
ponded to the invitation, 45 of which indicated a 
willingness to receive consultation (acceptors) and 7 
of which declined the invitation (rejectors). Non- 



* For purposes of this study a community general 
hospital is defined as a non-specialty hospital 
with no primary responsibility to the teaching of 
medical students but with a major obligation to 
fill medical needs of the community which it 
serves and in which it is located. 



respondents totaled 136, 



After a period of two months, follow-up letters 
were sent to all non -respondents , succeeded by a tele- 
phone survey conducted by PMI staff physicians to non- 
respondent hospitals in Massachusetts and Rhode Island. 
A low response rate from these two states prompted this 
telephone survey, the purpose of which was to obtain a 
proportionate representation from each state. 

Follow-up efforts resulted in a group of nine 
additional hospitals desiring to participate in the study. 
This brought the total group of acceptors to 54. 

Selection criteria were established to be applied 
to this acceptor group in order to ensure a representative, 
non-biased sanple of the study population: 

1, The number of sample hospitals '^elected from 
each state must be proportionate to the total 
number of "population** aospitals within the 
state . 

2, Bedsize distribution of the selected sample 
must be proportionate to the four state and 
national distribution, 

3, Selected sample must include a distribution of 
hospitals varying in distance from Boston, 
where both the consultative agency (PMI) and 
primary New England medical and educational 
resources were located. 

Upon application of the selection criteria, 40* 
hospitals were chosen to partici^jate as experimental 
hospitals. Distribution of the experimental group accor- 
ding to the three selection criter ia--stat e location, 
bedsize, and distance from Boston--may be found in 
Tables I, II and III respectively. 



* Although the original contract called for 30 
experimental hospitals, the sample number 
selected was increased to 40 in order to allay 
any research problems arising from the usual 
attrition of cases encountered in social science 
research . 



Tab:e 1 



Distribution of Experimtxta 1 Hospitals According to 
State Location as Compared with Distribution of Study 

Population 



STATE 


STUDY 
POPULATION* 


EXPERIMENTAL 
SAMPLE 




Number 


Percent 


Numb er 


Percent 


Massachusetts 


88 


46.8 


12 


30.0 


Maine 


53 


28.2 


14 


35.0 


New Hampshire 


31 


16.5 


9 


22.5 


Rhode Island 


16 


8.5 


5 


12.5 


Total 


188 


jlOO.O 


40 


100.0 



*Study population is defined as all short term stay, 
community gen: hospitals located in the srtates of 
Maine, Massa ..u^etts. New Hampshire or Rhode Island 
which were not currently and actively involved with 
other PMI activities relating to hospital -brised 
continuing medical educatic:' programs. 
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Table 2 
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Distribution of Experimental Hospitals by 
Compared with National and Study Population 


Bed Size As 
Distributions 




NUMBER BEDS 


NATIONAL* 


STUDY 
POPULATION** 


EXPERIMENTAL 
SAMPLE 






Number 


Percent 


Number Percent 


Number 


Percent 




Under 25 
25-49 


523 
1500 


8.6 
24.6 


4 2.1 
41 21.8 


1 
6 


2.5 
15.0 




50-99 

100-199 

200-299 


1528 
1147 
609 


25.1 
18.8 
10.0 


63 33.5 
46 24.5 
23 12.2 


16 
8 
9 


40.0 
20.0 
22.5 




300-399 
400 + 


325 
454 


5.3 
7.5 


7 3.7 
4 2.1 


0 
0 


0.0 
0.0 




Total 


6086 


99.9 


188 99.9 


40 


100.0 








Table 


3 






Distribution of Experimental Hospitals According to 
from Major Medical Center (Boston) 


D i s t anc e 




DISTANCE (miles) 




EXPERIMENTAL SAMPLE 




J 








Number 


Percent 




i a 
f 


0-49 
50-99 
100-199 
200 + 






10 
16 
9 
5 


25.0 
40.0 
22.5 
12.5 




1 

( 

( 


Total 






To 


100.0 





*Data obtained from J.A.H.A Hospitals Guide Issue, 1967. 
Includes all general, short-term hospitals, 124 hospital units 
of institutions, and 94 long-term, general hospitals. N.B. 
Because these statistics include long-term hospitals and 
university teaching and affiliated hospitals, data present an 
estimated reflection of national distribution of hospitals 
pertinent to this project. 

^*Data obtained f rom; J . A.H . A. Hospitals Guide Issue, 1967, 
Wilson G. Snillie, Jean A. Curran, ''Medical Care in the State 
of Maine. 1956-1962." fSponsored and financed by Bingham 
Associates Fund, Be thel ,* Maine , 1961); PMI surveys. 
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Project design called for a simultaneous selection, from the 
group of hosj^itals which accepted the invitation to participate in 
consultation, of a group ox control hospitals, that is, hospitals 
who would participate in evaluation segments of the study but would 
not receive consultation* The control hospitals would be matched 
as a group with experimental hospitals along the parameters of 
bedsize, state location, and geographic proximity to the ex- 
perimental hospitals* As with the experimental group, the control 
group was to include only those hospitals who had originally 
agreed to participate in the consul tati on* Due to the depletion 
of the acceptor group as a result of experiment a 1 group selection, 
only 13 acceptors remained • All 13 were selected for inclusion 
in the control group. Twenty-seven additional hospitals from the 
non-respondent category were selected according to the criteria 
discussed above.* 

Letters were sent to the selected sample inviting them to 
participate as a control in the study and stating the activities 
in which they would be involved. Participation would include a 
half-day interview by behavioral scientists of several key members 
of the hospital staff and completion of a few hospital bio- 
graphical forms for research purposes. The letter also indicated 
a time period during which the control hospital could expect a 
telephone follow-up from the consultative agency to obtain their 
response to the invitation and make arrangements for the interview 
visits. All but two of the hospitals, both of which had been* 
selected from the non-respondent cat egory , replied that they would 
be willing to participate as controls. Two replacements from the 
non-respondent group were invited and agreed to participate. 

Table 4 provides a visual description of the matched experi- 
mental and control groups distributed according to the selection 
criteria of bedsize and state location.. 



*To examine the possibility that the inclusion of 27 hospitals 
that did not respond to the initial invitation had introduced a 
sampl ing error, a series of in tra- group and inter-group analyses 
were conducted to test the appropriateness of the control group , 
Each of the two control sub-groups (13 respondents requesting 
consultation and 27 n on -respondents) was compared independent ly 
to the experimental group and to each other on the primary depen- 
dent measures of the study. When the two control sub-groups were 
compared with each other, no statistical ly significant differences 
were found between them. When each of the control sub-groups was 
compared with the experimental group, statistically significant 
differences of similar magnitude were found in both cases. These 
findings seem to indicate that hospitals which initially requested 
consultation were not more change-prone than those additionally 
recruited to serve as controls, and that the non -con suited hospitals 
constituted an appropriately matched control group. 
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Table 4 



Distribution of Experimental and Control Hospital Groups 
by Bed Size and State 



Bed Size 


MAINE 


(N=28) 


MASS 


. (N = 27) 


N.H. 


(N=18) 


R.I . 


(N =8) 


Exp . 


Contro 1 


Exp . 


Control 


Exp . 


Contro 1 


Exp . 


Contro 1 


under 25 


1 


1 


0 


0 


0 


0 


0 


0 


25-49 


4 


3 


1 


1 


0 


1 


1 


0 


20-99 


5 


6 


4 


4 


6 


5 


1 


1 


100-199 


1 


3 


4 


5 


3 


3 


0 


1 


200-299 


3 


0 


3 


4 


0 


0 


3 


1 


300 + 


0 


1 


a 


0 


0 


0 


0 


0 


Total 


14 


14 


12 


15 


9 


9 


5 


3 



An alphabetical listing of experimental and control 
hospitals with their locations may be found in Appendix 
D, Phase I, p.D-1. The reader is also referred to the 
map on p.20a^b which indicates the location of experimental 
and control hospitals for Phase I and Phase II. 

RECRUITMENT AND SELECTION OF CONSULTANTS 

In order to help ensure the provision of a consul- 
tation that was consistent with PMI's principles of hos- 
pital-based education program development , and to ensure 
standard izat ion of the consultation being provided to 
hospitals for study purposes, the recruitment of con- 
sultants was of crucial importance. The possibility 
existed for using either university-oriented medical 
educators or community hospital practitioners as consul- 
tants. Ideally, consultants should have been authorities in 
both areas --medical education and community hospital 
practice--but availability of such individuals was limited. 
The academicians would be expert in medical education and 
could provide ready access to teaching center resources, 
but would have to be trained in developing skills and 
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HKP HOSPITAL 
ff 

1. Alexander Castnan 

2. Athol Menorial 
5. Augusta General 

4. BarnstAblc County 

5. Bath Menorial 

6. Blue Hill Mcnorial 
7* 8on Secours 

I. 8ro<:kton 
9. BurbanL 

10. Cable Menorial 

11. Cape Ccd 

12. Cary y.enorial 

13. Central Mair.e General 

14. Charles Choate 

15. Chelsea Mer.orial 

16. Clarcnont 

17. Clinton 

It. Connunity General 

19. Concord 

20. Eastern Maine General 

21. Ellsabetn Ann Seton 

22. Elliot Connunity 
25. Emerson 

24. Exeter 

25. Fairview 

26. Falaouth 

27. Fogarty Menorial 

28. Franklin 

29. FranLlin County Memorial 

50. Frisbie 

51. A. R. Could 

52. Harrington Menorial 

55. Haverhill Municipal 
54. Henrietta Goodal 

5S., Hei ry Heyvood Memorial 

56. Holden District 

57. Holyoke 

58. Huggins 

59. J. A. Taylor Osteopathic 

40. J. 8.^ Thonas 

41. Kent County 

42. Littleton 

45. Maine Coast Metaorial 

44. Mary A. Alley 

45. Mayo Menorial 

46. Melrose-WaLei ield 

47. Menorial 

48 . Menori al 

49. Mercy 

50. Millinocket 

51. Milo Conr.unity 

52. Monadnock 

S5. Moore General 

54. Kashua Menorial 

55. N'ewport 

56. Kotre I)ase 

57. Notre Dane do Lourdes 

58. Osteopathic General 

59. Osteopathic of Main* 

60. Peoples benevolent 

61. Regional Memorial 

62. : Rumford 

65. Sacred Heart 

64. St. Joseph's 

65. St. Luke's 

66. St . Luke * s 
. 67. St. Mary's 

68. Salen 

69. Sceva Speare 

70. Scott Webb 

71. Sebasticook Vall«y 

72. Sturdy Menorial 
75. Synnes 

74. Thayer 

75. Tcbey 

76. Tri-County Osteopathic 

77. V. A. Manchester 

78. Waldo Cour.ty General 

79. WaterviUe Osteopathic 

80. Kebber 

81 . Kentworth^ Doug lass 

82. Wesson 
85. Westerly 
84. Woonsocket 



KEY TO HAP 
LOCATIO«S 

Derry, S.H. . 
Athol . Mass. 
Augusta, Main* 
Pocasset . Mass . 
8ath, Maine 
Blue Hill, Main* 
Methuen, Mass. 
Brockton, M.iss. 
Fitchburg, Mass. 
Ipswich, Mass. 
Hyannis , -Mas s 
Caribou. Maine 
Lemston, Vaine 
Koburn, Mass. 
Chelsea Mass . 
C 1 arenonw , .N' .H . 
Clinton, M.ass. 
Fort Fairt*ield, Maine 
Concord, .S'.H. 
Bangor, Maine 
Waterville, Main* 
Keene, .S'.H. 
Concord, Mass. 
Exeter, .S'.H. 
Skovhegan. Maine 
Falnouth* Mass. 
North Snithfield, R.I. 
Franklin, N.H.< 
Farnington, Maine 
Rochester, N.H. 
Presque Isle, Main* 
Southbridge , Mass. 
Haverhill, Mass. 
Sanford. Maine 
Gardner, Mass. 
Holden, Mass. 
Holyoke. Mass. 
Volfeboro. .S.H. 
Bangor. Maine 
Peabody, Mass. 
Warwick. R.I. 
Littleton, N.H. 
Ellsworth. Maine 
Marblehead, Mass. 
Dover-Fox craft. Main* 
Melrose. Mass. 
North Conway. N.H. 
Pawtucket , R.I. 
Portland, Maine 
Millinocket. Main* 
Milo« Maine 
Peterborough , N.H. 
Goffst'>wn, N.H. 
.Sashua. N.H. 
Newport. R.I. 
Central Falls, R.I. 
Manchester, N.H. 
Cranston . R . I 
Port land , Maine 
Fort Kent. Maine 
Brunswick , Maine 
Runford, !!aino 
Manchester, N.H. 
Providence, R.I. 
Kew Bedford. Mass. 
Fittsfield, Mass. 
Lewiston. Maine 
Salen, Mass- 
Plymouth, N.H. 
Portland, Maine 
Pittsfield, •!aine 
Attleboro, Miiss. 
Arlington, Mass. 
Waterville. Main* 
Warehan, Mass. 
Kittery, !!aino 
Manchester, .S.H. 
Belfast, Maine 
rfatervi 1 2e , .^lain* 
Biddoford, Maine 
Dover, N.H. 
Springfield, Mass. 
Westerly, R.I. 
Woonsocket , R* I • 



STUDY STATUS 



Cont ( 
Cont . 
Cont . 
Exp . 
Cont 
Exp. 
Cont 
Cont 
Cont 
Exp. 
Exp . 
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txp. 
Cont 
Exp . 
Cont 
Cont . 
Cont . 
Exp . 
Cont t 
Cont . 
Exp . 
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Cont . 
Exp . 
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Exp . 
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Cont . 
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attitudes needed to relate effectively to community 
hospitals. The practitioners, familiar with the com- 
munity hospital, would possess ability to relate to 
community hospitals, but would have to be trained in 
developing knowledge of education processes. PMI 
chose to exploit the already developed educational ex- 
pertise of medical academicians and launched a recruit- 
ment program accordingly. 

Three basic criteria were established for recruit- 
ment purposes: 

1. The consultant should be familiar with medi- 
cal education philosophies and methodologies. 

2. The consultant should approve of the concept 
that continuing education for physicians can 
be effective in a community hospital setting. 

3. The consultant should be familiar with the 
medical education and patient care activities 
and problems unique to each of the four 
states in the study. 

Thirteen consultants were recruited: one consul- 
tant whose background included many years of practicing 
medicine in the state ot Maine, one consultant from 
Rhode Island , and the remaining 11 from Massachusetts . 
Attempts were made to recruit a consultant from New 
Hampsh ire , but these attempts were not successful. All 
but one of the selected consultants had an academic 
position at a medical school. A full listing of con- 
sultants with their academic titles can be found in 
Appendix E, Phase I, p. E-1. 

ASSIGNMENT OF CONSULTANTS TO EXPHIlIMENTAL HOSPITALS 

In order to minimize the potential biases resulting 
from interaction of individual consultants with their 
hospitals, the following consultant assignment criteria 
were established: 

1. Assignment of consultants to a minimum of 

three hospitals. A larger hospital assignment 
was desirous but the necessary time commi tment 
that would be required could not be obtained 



from the consultants. 



2 Assirnnent of consultants to a distribution of 
hospitals varying in bcJsize and state location. 

3. Assignment of consultants to a distribution of 
hospitals varying in distance froni the consul- 
tant's home base. (It was felt that the distance 
between hospital and consultant's home base may 
affect the consultant's potential time and 
service afforded his hospitals.) 

Ten of the 13 consultants* were willing to consult 
with a minimum of three hospitals. Six of the ten 
^ccep^cS ai assignment of four hospitals. ^onsu ants 
were assigned a set of hospitals varying in b^dsi.e 
Trol llrll (over 200 beds) small (less than 50 beds) 
and located in at least two of the four states. 



ORIENTATION AND TRAINING OF CONSULTANTS 

As discussed in the study design section, the major 
independent variable r^anipulated ^^Jhe investigator in 
the present study was that of consultation. The primary 
au?s?ion to b3 answor-ed was, "What was the effect of 
2onsuUation cn the aevelopment of continuing Physician 
consuiLd ,„ follows that a major effort 

Sourd iave' ^be expended ?o'minimize bias and -ximize 
^el abiUty of the consultant's approach to the hospital, 
In essential step in that direction was taken m the 
selection'of consultants and their assignments to exper- 
iminfai hospitals. More funda.nental , however, was the 
adequate indoctrination and preparation of co"^"^^"^^^ 
In TrTer Jhat they might provide analogous 
services to hcspitalc.. This necessary component was 
acconp!ished in three consultant training sessions as 
^en as individualized informal debriefing and review 
sessions held with the PMI staff. 



As require-ents of the consultation commitments became 
fuUy recornized. three, consultants found it possible 
^0 ior^ wUh onl^ one or two hospitals. Due to the 
time specifications of the contract, it was not feasibl. 
iidrop these consultants and recruit others to replace 
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Formal Training 



The first training session, a two-day orientation 
workshop (June 27-28, 1967) was helJ for the consultant 
staff shortly after final selection of hospitals and 
consultants, and prior to the consultants' going out 
into the field. (A copy of the agenda and minutes of 
the meeting may be found in Appendix F , Phase I, ppJ^l 
a-h.)The main objective of this session was to orient 
both PMI staff and consultant personnel in the follow- 
ing areas: 

1. The project and contract objectives, 

2. The project design, 

3. The role of both staff and consultant per- 
sonnel, and 

4. Implementation of the consultation process. 

During this initial session, the nature and purpose 
of the present study was discussed, and the background 
of PMI and its relation to other r,edical institutions 
were reviewed. The most important part of this initial 
session, however, was embodied in areas numbers 3 and 4 
above. A large portion of the two-day session was devoted 
to 1) clarifying the goal of consultation and the role 
of the consultant; that is, "to help hospitals to develop 
their own continuing medical education programs- -not to 
establish programs for them"; 2) indoctrinating them m 
detail as to the content and process of consultation; 
and 3) briefing them as to what was expected of them by 
both PMI and the hospital. 

Consultants were expected to make two visits to 
each of their assigned hospitals. "Reports of visits" 
consisting of what occurred and what was discussed with 
hospital personnel during the visit were to be submitted 
to PMI after each visit . (Appendix F, p. F-li). 

The purpose of the first visit was multifold: 

1. To explain PMI's purpose and the nature of the 
contract. 

2. To state objectives of the consultation, that 
is, what the hospital should expect from PMI, 
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and what PMI expected from the hospital. Con- 
sultants were to explain that PMI would assist 
them in establishing for themselves a continu- 
ing edu cation program. Cooperation of the 
hospital would be needed in periodically com- 
pleting various forms, when warranted . 

3. To ask the hospital to appoint a physician 
liasion between PMI and the hospital. Such an 
individual could be either a DME, chairman of 
the education conraittee, or any staff represen- 
tative who would act as the local education 
coordinator and who would be able to assist 

the consultant in assessing needs and identify- 
ing resources in the hospital. 

4. To outline a plan of action to be taTcen by the 
hospital before the consultant's next visit: 

a. Enlist support of the local education 
coordinator to summarize the state of 
existing education activites and hospital 
facilities nnd services. Such data would 
serve a two-fold purpose: 1) as a take-off 
point for discussions of needed improve- 
ments in the education program, and 2)* as 
a pre-mi-asurc for the evaluation of 
changes in CME program developnent before 
and after consultation. See Appendix G, 
pp. G-1-7 for a copy of forms used to 
gatlior the information. These forms had 
been used by PMI in its program development 
activities prior to the onset of the present 
contract. They are discussed more fully 

in the section of this report entitled 
"Data Collection", pp. 31-33. 

b. Speak to staff representatives to gain 
acceptance of PMI endeavors aimed at deter- 
mining njeds, interests, and resources of 
the medical staff as procured by "staff 
questionnaire forms". (See Appendix G, pp.GS 
a-b for copy of form, and section entitled 
"Data Collection" for a more complete dis- 
cussion of the questionnaire.) 

At a meeting held on September 12, 1967, presenta- 
tions and discussion focused on content and process of 
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consultant visits, as well as desired format and scope 
of recommendations. Most consultants had conpleted at 
least one visit to hospitals. The following areas were 
discussed:* 

1. Nature and content of visits to hospitals; 

2. Role of hospital physician education coordi- 
nator in the project; and 

3. Nature and scope of recommendations. 

Recommendat ions were to be clearly and 
explicitly stated, avoiding ambiguity. 
Reference was to be made to specific rather 
than to general areas, for exaniple, "Establish 
an ICU unit" rather than "Establish more 
facilities and services." 

The following types and content areas of 
recommendat ions were reviewed : 

a. Oral vs. written 

As noted previously, all consultants were 
required to transmit a set of wr i t ten 
recomm ^ndations to the hospital upon 
completion of both consultation visits. 

While on visits, however, a number of 
consultants made oral suggestions to the 
hospital which were not included ultimately 
in the written recommendations transmitted. 
In order to assure valid verification of 
the total impact of consultation, it was 
deemed necessary by the research staff to 
discer-- and denote any such suggestions 
made. Such advice or suggestions were 
cons idered *'oral recommendat ions'* . 

b« Educational and non-educational 

It was not possible to limit recommendations 
to strictly educational matters. To be 
class i fied as educational , a recommendat i'^n 
had a primary objective of affecting the 
education of hospital per:>onnel, e.g., 
"implement grand rounds," "initiate a 



* (Appendix F, p. F-2a includes agenda.) 
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team approach to educate programs 
centered on the coronary care unit , " "medi- 
cal staff and administration jointly 
appoint a DME" . Any recor.ir.iendation that 
did not specify a primary educational 
objective or wr.s not directly related to 
the education of hospital personnel was 
placed in the non-education category, e.g., 
"develop an ICU," provide salaried emer- 
gency ward staff coverage," "investigate 
the possibility of providing home care 
services . " 

c. Physician and n on -physician professionals 

Ir accordance with the belief of PMI that, 
whei> appropriate, a continuing medical 
education program should be multi-disci- 
plinary, that is, include nurses and other 
health professionals, consultants were 
encouraged to formulate recommendations 
addressed to the continuing education of 
n on -physician personnel . 

In order to assist the consultant in developnent of 
rcconmendations , and to standardize the con.^ult- ".ion 
process, consultants were furnished with a guide fcr 
preparing reconmendat ions and a set of procedures 
and suggested format for transmitting; recommendations 
(see Appendix F, p. F-2b-c). 

Also included at the September meeting was a pre- 
sentation by the project staff of resource raterial 
available for consultant use. Information relaxing 
to hospital libraries and tlic Piofessional Activity. 
Study and Medical Audit Program (PAS-MAP) was discussed, 

After Phase I consultation visits were terminated and upon 
completion of the evrluation follow-up interviews, a f^^^^l 
consultant meeting was held on April 25, 1968 . (Appendix P ; 
p. F-3 contains agenda). The purpose of this final nieeting 
was twofold: 

1, Present preliminary findings of Phase I research 
(the reader is referred to section of prcivint report 
dealing with Results, of Phase I, for a thorough 
discussion of the findings); and 

2. Discussion of and orientation to contract extens-^on 
(Phase II). 
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Informal or Individual Training of Consultants 

' addition to formal consultant meetings , each 
medic 1 consultant received advlitional individual 
assistance and guidance in the following manner: 

!• Re view of "report of visits" 

As noted prevxoi .ly, all consultants were 
required , upon completion of each hospital 
visit, to submit tc the PMI staff a "report 
cf visit" sumnarizing areas discussed with 
hospital personnel and their reactions to or 
thoughts regarding the visit or the hospital • 

Each report was careful ly reviewed by the PMI 
staff. The prir.r.ry purpose of the review 
was to provide background and /or supplemental 
information to the consultant. For exampl e, 
if a specific request was made for information 
concerning a r-^ob lem area, the staff would 
provide the consultant with further assistance 
in the specif i e-.;. area , ■ 

2 . Review of pro;posed recommendations 

Prior to the cransmittal of recommendations 
i toahospital,anin-pers on review session 

was held with each consultant. The review 

commit tee was- composed of the project director, 
! a medical sta£f member, and a research staff 

member. 

1 The purpose of the session was to further in- 

struct consul t ants in the techniques of edu- 
cation program development on a personal basis 
! using the case study approach. Also it was 

i ' intended that the^e sessions would provide 

addi t i< i guidance to consultants in formu- 
lating IS final recommendations. 



I j CONSULTATION IMPLEMENTATIO.N * 

All 13 consultants vj.sited their hospitals during 
I the summer and fall of 1967 following the procedure 

I outlined in the June :i7-28 training workshop, pp. 
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*To UbbisL Liie rcauci j. i. .u'* d.c l c» l a u u jl 1 1 g 1 1* c I'l a t U 1* e a u d p 1* O - 

cedure of consultation, case studies of two typical experi- 
mental hospitals arc presented in Appendix I. The case 
studies include consultation activities for both Phase I 

and Phase 1 1 . 
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Consultation sessions usually lasted half a day to a full day. 

Discernable deviations from the consultation 
plan were the use of forms, the time schedule and the 
numbeir of visits hospitals received. Twenty-two of the 
40 experimental hospitals did not return tlie Physician 
Activity Sheet and 10 of the hospitals submitted less 
than 50 percent of the completed staff questionnaires. 
Reasons for low return rates varied from consultants 
choosing not to submit the forms to the hospitals for 
completion, to hospitals failing to complete the forms 
after receiving them from the consultants. 

Consultation visits, as noted in the Plan of 
Progress, page 15, we re. sc. :?duled to appear 
during the months of August, September and October cul- 
minating with a set of written recommendations which 
were to be sent to the hospitals at the beginning of 
November. Fourteen hospitals received one visit from 
their consultants, the remain ing 25 receiving two or 
three visits. Some consultants began visiting their 
hospitals in July when it became apparent that the 
summer months, a time vhen a large proportion of nedical 
staffs are vacationing, would not necessarily be a hin- 
drance to the consultation rocess. Several hospitals 
received consultation visits after the November 1 dead- 
lin3 for transmission of recommendations to hospita^ls. 
Twenty hospitals received their recommend at ions after 
November 1, the latest date of transmission being mid- 
December, and three hospitals did not receive them at 
all. In order to allow hospitals to have approximately 
the same amount of time to implement r ecommendat ions , 
the follow-up interviews for those hospitals receiving 
late recommendat ions v;ere scheduled for the latter part 
of the follow-up period. A total of .355 rqcommendat ions 
were submitted to 57 hospitals. The nature and outcomes 
of the recommendat ions are discussed in the Results sect ion. 

DATA COLLECTION AND RESEARCH INSTRUMENTS 

Two categories of data were collected: 1) data for 
purposes of program development; and 2) data for 
research and evaluation of the impact or effect of the 
consultation process. 

Data Collection for Purposes of Program Development 



As discussed under the section of the present 



report entitled "Study Objectives", the consultation 
procedure for assisting comnunity hospitals was based 
on first ascertaining the needs and interests both of 
the hospital and the nedical staff on which a CME pro- 
gran could be developed and identifying local resources 
for the program. Instruments used to facilitate collec- 
tion of data pertinent to the above objectives had 
been extensively utilized by PMI in consultation acti- 
vities prior to the onset of the present contract. The 
reader is referred to Appendix A, "Positive 
Approaches to Continuing Medical Education in Community 
Hospitals," by Xorman S. Stearns, M.D., for a discussion 
of PMI ' s ph i los ophy of consultation, and rationale 
underlying zhe use of the forms. (Copies of the instru- 
ments may be found in Append i x G , pp . G 1 - 8 . ) 

!• Schedule of Educational Activities 

In keeping with the philosophy of an inter- 
disciplinary approach to CME program develop- 
ment, two forms were provided-- one for 
physician e ducat ion activites, a second for 
n on -physician education activities. (Appendix 
G, pp. G2 - 3.) The purpose of soliciting 
the above information was to provide a sum- 
mary of existing activities at the hospital 
in order that the consultant could better 
understand the existing status of the educa- 
tion program and that he with the hospital 
could begin to determine some of the inade- 
quacies of the program. 

2. Hospital Facilities and Services 

Hospitals were to complete the "Hospital Faci- 
lities and Services" form (Appendix G, p.G-5) 
by checkin?] those facilities and services 
available at the hospital. This data was to 
be used to determine the potential implica- 
tion of facilities and services as a basis 
for educational programming , e.g., hospitals 
with ecu. units have the potential of building 
a segment of their continuing education 
activities program around the CCU un i t . 

3» Staff Questionnaire* 



In order to ascertain the educational needs 



and interests of the staff physicians, a ''Staff 
Questionnaire" instrument (Appendix Q, p . G-s) 
was utilized. The purpose of the instrument 
was to assist the consultant in deternining 
resources, caliber, nnd interests of tlie hos- 
pital medical staff regarding continuing 
medical education. The form was to be completed 
by each member of the active staff and then 
forwarded to PMI for compi lat ion and summariza- 
tion . 

4. Key Considerations in Development of Educational 
Programs --Tent at ive Reporting Sheet for 
Consultation 

This checklist of items (Appendix "G, pp. G-ba-d) 
provided a guide for the personal use of the 
consultant. In essence, the instrument was a 
debriefing form enabling the consultant to 
indicate areas discuss ed with the hospital 
team during the two consultation visits. The 
value of the instrument was twofold: 

a. It facilitated report writing, i.e., 
"reports of visits", for the consultant 
and^ 

b. By enumerating specific areas discussed, 
it provided a more thorough and more 
accura t e account of the impact being made 
by the consultant. 

5. PMI Planning Calendar 

The chronological time record (Appendix p.G-7) 
depicts days, data, and months of the year 
and denotes all legal holidays and religious 
observances. For each day of the year, an 
area is provided in which to insert scheduled 
or planned educational activities. This 
instrument was to encourage s chedule planning 
of educational activities for long periods of 
time and avoid conflicts with ho lid ays and 
other "no meeting" days. Planners could also 
view time and c^Dnt ent aspects of the program 
in perspective for an entire year. 

All hospitals were provided with the appropri- 



ate forms; however, it was the responsibility of the indi- 
vidual consultant to encourage collection of data* Phy- 
sician education coordinators, or other designated 
individuals , were responsible for actually collecting 
the information. 

Data Collection for Purposes of Evaluation 
and Analysis 

It will be recalled (see section entitled "Study 
Design**) that a pre-post design, and an exper iment al - 
control design were employed in the present investiga- 
tion for purposes of evaluation. In order to determine 
the extent to which the consultation program influenced 
education program development, attempts were made to 
collect data before and after consultation in experi- 
mental hospitals. Data collected as discussed in the 
preceding section for purposes of program development 
had a two-fold purpose in that information also, theo- 
retically, was to serve as a pre-measure, i.e., what 
experimental hospitals were doing prior to the inter- 
vention of PMI consultants. However, consultants did 
not systematically use the data collection forms at the 
onset of consultation, (see Consultation Implementation, 
p. 29 therefore making ;t impossible to establish base- 
line data for evaluation purposes. Consequently, this 
aspect of the study design, that is, deternination of 
the impact of consultation by ;*-naly2ing differences in 
pre and post measures cn educational program development, 
had to be abandoned as an evaluation strategy.* 

Post (or after consultation) measures, collected 
in both experimental and control hospitals, following 
the c'esignatcd ^'Recommendat ion Implementation Period'', 
are the sub j ect of discuss ion in the present section, 
as is the collection of mediating factors (facilitative 
or inhibitat ive) affecting implementation of continuing 
physician education programs. Types of data collected 
are delineated, and methods of data collection are 
discussed. 

Types of Data Collected 

As discussed in foregoing sections, '^change" data, 
that is changes occurring in continuing medical educa- 
tion due to the impact of our consultation, constituted 
the major dependent variable. This ''change data" was 
operationally defined as: 



♦Attempts to reestablish baseline data during the 

v»UocA nf thA ctiidv will be discussed in Phase II. 



1. Degree and rate of implementaiion by experi- 
mental hospitals of the recomncndati ons 
transmitted by consultants. 

2. Alterations of pre-existing conditions which 
were initiated by and occurred within both 
experimental and control hospitals after 
completion of the consultation program. 

Independent variables were categorized into two 
areas, manipulated and fixed. The independent variable 
consultation, was manipulated by the investigator. Fixed 
independent --.riables for Phase I analysis were cate- 
gorized into four areas: 

1. Hospital Characteristics 

Bioc-aphical data about the hospital and its 
environment, e.g.. the nunber of beds, proximity 
to r.edical teachi g centers. (See Appendix M. 
pp. H-la-c) 

2. Staff Characteristics 

Objective data about staff of hospital (adminis- 
trative and medical), e.g.. degree-holding 
administrator, age of medical staff .. percent 
of G.P.'s to specialists, etc. (See Appendix 
H, p. H-lb for' full listing.) 

3. Interviewer- Interviewee Characteristics 

e.g., number of interviewers, number of inter- 
viewees, etc. 

4. Consultant Characteristics 

e.g.. attendance at orientation and training 
sessions, transmittal of recommendations on 
tine, etc. 

Number^ three and four above are a special type of 
independent variable and are technically known as inter- 
vening or co nditioning variables. The rationale under- 
lying the collection of "consultant" and "i.nterviewer- 



interviewee'' data derives itself from the methodological 
fact that uncontrol led-f or variables associated with 
both consultation and interviewing, e.g., differential 
attributes of consultants and interviewers, etc., may 
have a contaminating effect of the entire change process 
(or report of change). A relationship between variables 
is likely to be spurious, if it is due to or caused by 
intervention of soiTie third "contaminating" variable or 
measurement. Therefore inclusion of the above variables 
in the measuring process serves as a means for taking 
care of possible uncontrolled-for variables arising 
from unwanted sources. 

Method of Data Collection 

Data was collected in several ways for evaluation 
of the consultation program: 

1. Documented source material, e.g., examination 
of records, reports, etc. 

2. Questionnaire instruments 

3. Mail and telehpone solicitation 

4. In-person interviews 

The first three methods were employed primarily for 
collection of independent variables, while the latter 
method was engaged for collection of data pertinent to 
dependent measures. 

Illustrative of the first three methods are sources, 
instruments, and procedures such as the following: 

1. Documented source" material 

a. Journal of the American Hospital Association 
Guide Issue j 967 served as a source of 
material for data regarding hospital charac- 
teristics, e.g., bedsize, type of control, 
residency and internship program, facilities 
and services, etc. 

b. Various consultant characteristics data 
were gleaned from the consultants* curricu- 
lum vita. 



2. Questionnaire instruments 

The "Physician Staff Questionnaire" (discussod 
above under section entitled "Data Collection 
for Purposes of Program Development") yieldv'd 
data on medical staff attributes such as age 
of staff, size of active staff, specialties 
of staff, etc. Appendix G ,p . G-S contains a 
copy of the instruments. Sections 1 through 
12 of the questionnaire were used as source 
material . 

3* Mail and telephone solicitation 

Used primarily for re-solicitation of materials 
from non-respondents. 

In addition to sources discussed above, other itemized 
sources were of value in gleaning additional needed in- 
formation. For example, mileage figures for "sphere of 
influence" data were obtained from road maps equipped 
with mileage figures. 

Although data for independent variables were gathered 
from the three sources as discussed above, the "Hospital 
Attribute" form (Appendix H, pJl-1) was the prinary 
research form utiliz-:d for recording of independent 
variables. Inspection of the form permits one to view 
data for hospital characteristics (I), nedical staff 
characteristics (II) and consultant characteristics 
(IV). The reader will note a category entitled "Socio- 
economic characteristics of the community in which the 
hospital is located" (III) . This independent variable 
was later eliminated from the study due to invalid 
and inconsistent findings of data, e.g., source material 
differed within each state. Upon full utilization of 
the above sources and recording of such on the "Hospital 
Attribute" form, the instrument was brought to the inter- 
view scene at the time of the follow-up evaluation visit 
in order to verify or correct obtained information and 
to supply missing information. 

4. In-person interviews 

At the conclusion of the "Recommenda t ion ' Per iod" , 
behavioral science interviewers, trained and 
prepared by PMI, conducted "follow-up visits" 
to each experimental and control hospital in 



order to collect data regarding the dependent 
variables : 

a. Changes occurring at both experimental and 
control hospitals during the consultation 
per i od . 

b Action taken by experimental hospitals on 
recommendations transmitted by consultants, 
i.e., accept, reject. 

c. Supplemental data for background charac- 
teristics of independent variables. 

Visits to hospitals by interviewers were 
scheduled for February 15-March 31, 1968, four 
months after recommendations were transmitted. 



Recruitment and Training of Interviewers 

Behavioral science oriented professional inter- 
viewers were recruited by PMI in order to assist in 
conducting follow-up visits. Doctoral candic.ates of 
the major universities in the area served 
:naterial for recruitr.ent . Final selection of/^^^i^f J! 
pivoted on their degree of prior experience in the tech 
niques of observation and interviewing, in addition to 
their knowledge regarding a medical setting. 

Ten interviewers, five from the PMI research staff 
and five recruited from outside sources, were selected 
to visit the 40 experimental and the 40 control hospi- 
tals In order to so::iewhat. e 1 ininate interviewer bias, 
individuals were assigned to both experimental and con- 
trol hospitals. Assignment was based mostly on geo- 
graphic proximity of hospitals to each other, so that 
interviewers could reach several in one trip. 

Training of interviewers was accomplished in two 
training sessions of three hours each. Method of 
training consisted or: 

1. An orientation to PMI and the contract 



Orientation consisted of a review of project 
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objectives, a discussion of the philosophy and 
progress of the project, and the purpose of 
the interview visits. 

2. Preparation for hospital encounters 

• 

In preparation for an actual interviewing 
situation , individuals were introduced to 
the workings of a "typical" connunity hospital 
sett ing . Simulated intervievcs were organized 
by the research staff as practice sessions. 
Brief case presentations were introduced, and 
prospective interviewers were afforded an 
opportunity at rcle playing an actual inter- 
viewing situation . 



3. Individual attention 



Each interviewer was presented with a packet 
of materials consist ing of information relevant 
to the specific hospitals to which he was 
assigned. In addition to the necessary research 
forms, included in Appendix H, pp .H-1-4 and 
discussed in a subsequent section on "research 
instruments utilised by the interviewer",* 
iiraterials ir eluded: a listing of the hospitals 
for which the interviewer was responsible, 
scheduled dates of interviewer visits, a set 
of travel directions to each hospital, and a 
set of directions for interview procedures. 
(The reader is referred to Appendix H , p . H-2 
for a C9py of set of directions distributed.) 

The Hospital Interview 

Extensive evaluation interviews were conducted at 
each of the 80 hospitals taking part in the study. 
Differing, though parallel, interview designs were 
used in the experimental (consulted) and control (non- 
consulted) hospitals. Individual interviews* were 
conducted with rive people in the consulted hospitals: 
physician education coordinator, hospital adninistrator. 



*In a small percentage' of cases, group interviews 
were held at the request of the hospital. 
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president of the medical staff, director of nursing, and 
a represertative fron the board of trustees. (The latter 
two individuals were not interviewed routinely but only 
when available.) On the other hand, in the non-consu 1 teJ 
hospitals interviews were conducted with only two people: 
hospital adninistrator and president of medical staff 
or a substitute , * Intervi ews lasted approximate ly one 
hour for the experimental hospitals and 45 minutes for 
the controls. All interviews were completed within a 
three month period of time. 

Interviews in both consulted and non-consulted 
hospitals focused on obtaining data regarding: 1) 
changes** having occurred during the ''Recommendation Imple- 
mentation Period" and 2) verification and supplemen- 
tation of information on the "Hospital Attributes" form. 
In the experimental hospitals, additional inf ormat i on 
was gathered regarding the action taken by the hospital 
on recommendations which had been submitted by the 
consultant four months previously. 



Research Instruments Utilized by Interviewers 

The following instruments were utilized by the 
interviewers for collecting the data. 



*Becauso PMI solicited cooperation and effort 
from the non- consu 1 t ed hospitals without providing any 
immediate service or benefit to them, it was deemed 
necessary to minimize the burden of the hospitals ' time 
commitment. Consequently .the research staff requested 
onl^ the two interviews in contrast to the three-five 
interviews at con suited hospitals. It was assumed that 
the two inter view designs were capable of producing 
comparable data. Statistical tests of this hypothesis 
proved it correct, i.e., there were no significant 
differences in the number of reported changes as a 
function of the number of persons interviewed. (t=.88) 



**Change in community hospital continuing medical 
education programs . 
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1. "Interview Guide Sheet" 

The ''Interview Guide Sheet" (Appendix H, pp, H3a-b 
was used both in expcrir.ental and control 
hospitals to elicit in for nation relevant to 
*'char. f;es" occurrinj; within both groups of 
hospitals during the four montli interval 
designated as "Recommendation I implementation 
Period". Use of this form served to standar- 
dize the reporting process-- thus ensuring a 
7 nore valid and reliable collection of data. 

The first question, an open-ended Oi*e, was 
desii:ned to elicit responses in a non-re- 
st«.*icted fashion: "What changes have been 
initiated?" Subsequent questions related to 
specific areas of hospital functioning, e.g., 
budget, facilities and services, etc. Ques- 
tions attempted to invest igate all aspects of 
hospital functioning. 

2. "Recommendation Review" Form 

The "Recommendation Review" form , (Appendix H-4)which was 
used in exper iriontal hospitals only, embodied 
a listing of the recomr.iendat ions sent by the 
consultant to the hospital. The purpose of 
the form war. to determine the implementation 
status of recor-nendations • On attached 
sheets next to the listing of recommendations, 
space was prov ided for recording (by the 
interviewer) of "ctions taken by the hospital 
relating to recommended items. Three possible 
classifications of actions were indicated: 1) 
Yes -> - denoting acceptance and implementation 
of r e commend at ion; 2) -denoting rejection; 
and 5) Partial I m p 1 e i a t a t i on indi eating 
some action taken but not fully operational. 
A separate attached sheet was used for each 
individual interviewed, 

3. "Hospital Attribute" Form 

As discussed previously, available data for 
independent variables were recorded by the 
research staff on the "Hospital Attribute" 
forr. (Appendix H, pp H-la-c). The instrument 
was then brought to the interview scene in 
order to verify or correct recorded information. 




Having collected all pertinent data at the inter- 
view scene, the interviewer returned the forms to PMI. 
A shor*, meeting was held with each interviewer in order 
to review hospital reports and discuss any p rob lens 
encountered • 

Data collected by the interviewers sere rece ived 
by PMI prij?*arily in the form of the response given by 
the interviewer. It was the; task of the research staff 
to tr an slate the response into measurable terns, that 
is, to process the data--the discussion of which is the 
focal point of the forthcoming section. 

PROCESSING OF DATA 

Processing of data refers to those steps and or 
measures that are taken in or 'er to get the data ready 
for analysis. Stated in other terms, data processing 
translates raw data into quantifiable or descr ipt ive 
measures for analysis. 

Before discussing ^[ata processing, which was done 
throughout Phase I at varioii.s steps, it seems worth- 
while to digress for a momen". r.o reiterate and summarize 
the development of the study to this point: research 
objective of the study wis to assess and evaluate 
changes in r' e continuin;; m'^cical education pro<;rams of 
community hospitals resulting iron a series ov consul- 
tations by visiting medical educators. Changes con- 
stituted the major dependert variables and were opera- 
tionally defined as stated in the section "Data Collec- 
tion and Research Instruments", p, 32, Two types of 
change measures emerged: those collected in both ex- 
perimental and control hospitals; and those collected 
in experi mental ones only. 

It is the purpose of the present section to inform 
the reader or the process applied to the data in order 
to "ready" it for analysis. The nature of raw data 
collected will be discussed as will transformation of 
this data into measurable torns. The dependent measures 
will be presented first, followed by a discussion of 
the transformation of independent measures • 
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Process in g of Ocpcntlcn t Mca urcs 



Chan?^c Measures Gathci^cd in Both llxper incrila 1 and 
Control Hosni lais 

The research instrument used to gatlicr chanp^ d^ta was 
the "Jnterview Guide Sheet" as discussed under a p. ' • 
ceding section of the present report. 

Separate sheets Cresp on ses) of data emerged for 
each person interviewed (five in experimental, tl ree 
in control). Raw data was in the form of approximated 
responses of tlic intcrvi ewee as to what had changed" 
in various areas of the hospital. 

The first task set before the research staff was 
to compile data elicited from the various interviewees 
at each hospital in order to determine any duplication 
of reported changes. That is, in some instances the 
same change was reported by a number of interviewees at 
a hospital. Care was tMen taken to record such duplicate 
reports as only one change. 

After having established the iiumber of "changes" 
occurring witliin the Ixcspital as determined by reports 
of interviewees, the research staff was tlien faced with 
the question of val id i ty- -were all "reported chances" 
"valid changes", that is, were tlicv tangible and quanti- 
fiable? 



"Reported changes" were to satisfy three criteria 
established ^y tlie research staff in order to qualify 
as a "valid reported change". Changes not meeting 
criteria were deleted. 

Criteria for "valid reported change^*' was as 
f ol lows : 

1. Reported change must be hospital related, 
i.e., changes occurringin the ccinmuni ty 
at large, for instance, development of a 
Head Start Program, was n .<t accept d* 



2m Reported change must manifest itself in be- 
havioral or physically measurable terms . 
Change in attitude was not accepted, unless 
accompanied by behavioral changes. 

3. Some manifestation of a reported change must 
have occurred within the preceding four month 
period of tine, i.e., "Recommendation Irople- 
mentation Period". 

Having established and verified all changes which 
occurred during the four month period in each specific 
hospital, the research staff was then able to classify 
"changes** into specific categories for analysis. 

"Educational" and "non-educational" changes con- 
stituted the major breakdown. Categories were opera- 
tionally defined as follows: 

Educational Change--any hospital-initiated change 
occurring withir the four month implementation period 
which directly related to the education of hospital 
personnel including physicians, nurses, allied health 
personnel and other. 

Non-Educr .^.ional Change--any other change that was 
i\ospital-init ia Lud and occurring within the four month 
implementation period, for example, purchase of patient 
care equipment, development of special care units. 

Categorized data were numerical ly totalled indicating 
total number of changes (educational plus non-educational) 
number of educational changes, and number of non-edu- 
cational changes . 

Change Measures 'fathered in Experimental Hospitals Only 

As discussed previously, the "Recomnendat ion 
Review" form was utilized by the interviewers in order 
to determine the implementation status of the recommenda- 
tions submitted to experimental hospitals . 



As in the case of the "reported change" uata, a 
separate set of responses emerged tor each person inter- 
vicKcd. However, only two to three int crvicKecs , ad- 
ministrator, physician education coordinator and at times 
the director of narsing (if the recommendation pertained 
to nursing education) were required to participate in 
this portion of the interview. Rcju* data was collected 
for each reconnendatioi> in the form of discrete re- 
sponses to three possible categories: yes, no, or 
part ial ly imp 1 em en ted . 

The major objective of data transformation was to 
determine the nature and rate of recommendation imple- 
mentation for each hospital. The first task set before 
the project staff was to determine the action taken by 
the hospital to each specific recommendation , i.e., 
implementation status, as reported by interviewers. 

In order to determine the implementation status 
(yes, no, partial) of each recommendation, the following 
criteria were established: 

1. When all individuals interviewed unanimously 
agreed upon the status of a recommendation, 
it was cl as sified accordingly. 

2. When agreement concerning a r ecomriendation 
vas not unanimous, the response of that 
individual whose hospital function most 
closely corresponded to the nature of the 
recommendation was used • 

Three degrees of implementation status were used: 

!• Full, or complete implementations-implementa- 
tion or adoption of recomncndat ion was com- 
pleted during the four month implementation 
period. Throughout the present report the 
term "I" will be used to refer to full or 
comp jete implementation. 

2, Partial implcmentation--process necessary 
to fully implement recommendation was 
started, but not necessarily completed 
during implementation period . 



Two varieties of partial implementation emerged: 
(1) instances in which recommendations were 
reviewed and accepted for further action, e.g., 
sent to a cor:i!?.itt ee and (2) instances in which 
recommendations were partially implemented but 
could not be completed within the alloted four 
month implementation period. The term "P" 
vill be used throughout the present report to 
designate partial implementation. 

3. Re jection-"hospital rejected or took no action 
on the recommendation. The terra "R" will be 
used throughout the pres ent report to designate 
re j ection. 

A further grouping of recommendations were those 
recommendations which were implemented to any degree, 
that is, fully or partially. The term given to this 
classification was "IPI" and denotes the toal number 
or percentage of recommendations which any positive 
action was taken. 

Implementation rates, i.e., degree of implementation 
of recommendations, were established for each hospital. 
The fol lowing measures , such as the hypothet ical one 
illustrated in Table 5, were determined. 

Table 5 

Hypothetical Exampl e 

Implementation Status of 20 Recommendations 
Transmitted by Consultant 



RECOMMENDATION NUMBER PERCENT 

STATUS 



Complete Implementation ("I*') 
Partial Implementation ('^P'*) 
Complete plus Partial 

Implementation (''IP!*') 
Rejected C'R") 



15 75% (15/20) 

2 10% (2/20) 

17 85% (17/20) 

? ..15% (3/20) 



I and IPI rates established for each individual 
hospital may be found in Appendix J, p.J-1. 

Recommendations were further broken doKn by type. 
Classifications consisted of educational and non-educa- 
tional and were operationally defined in the same way 
as hospital change measures. 

Educational Recommendation-any recommendation 
which directly related to the education of hospital 
personnel includir.g physicians, nurses, allied health 
personnel and others. 

Non-Educational Recommendation -.any recommendation that 
did not specify a primary educational objective, for 
example, purchase of patient care equipment, development 
of special care units. 

Thus, for each hospital the following sets of 
data were established: 

Total number of recommendations transmitted by 
the consultant to the hospital, e.g., 20 

Number of educational-- 16 

Number of non-educational--4 

More important, however, was the application 
of the breakdown of recommendations by type, i.e., 
:educational--non-educational , to the degree of 
implementation categorization. In keeping with the 
example given in Table 5, of the 20 recoramenaations 
transmitted by the consultant to the hospital, 15 
(75%) were fully implemented and 2 {10%) were partially 
implemented. This constitutes a "degree or rate of ^ 
implementation" categorization for total reconraendat ions . 
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Sixteen rcconraendations were educational in 
nature, and four were non-educational in nature. 
The educational and non>ecucational J^^o'"^^^"^^^ 
were further broken do;s-n ly degree of implementation. 
Table 6 illustrates this breakdovn. 

Table 6 
Hypothetical Example 

Educational and Non-Educational ^^J^^^^^^^'" ^y/JS^Con"^^' 
tion Status of 20 Recommendations Transmitted by Con- 
sultant 



RECOMMENDATION 
STATUS 


TOTAL 




EDUCATIONAL 


NON- 
EDUCATIONAL 




Numb.^r 


% 


Numb e r % 


Number % 


Total recommendations 
transmitted 


20 




16 




4 


Implementation : 












Complete ("I") 


15 


75% 


14 


85.5% 


1 25% 


Partial ("P") 


2 


10% 


2 


12.5% 


0 0% 


Complete plus 
partial ("IPI") 


17 


85% 


16 


100.0% 


1 25% 


Rejected 

9, 


3 


15% 


0 


0% 


3 75% 



In addition to categorizing recommendations in 
quantitative terms, the data were also prepared for 
non-quantitative, cont.nt analysis.* Content 
analysis Was performed on both reconmendat ions sub - 
mitted to the hospital and recommendations imple- 
mented by the hospital. In order to prepare the 
data for content analysis the following steps were 
undertaken : 

!• Specific categories were established to 
determine what "types*' of recommendations 
were made and inplencnted for each hos- 
pital. As discussed previously, the major 
typology breakdown was educationa 1 and 
non -educational . 

2. Data transformation of the study's depen- 
dent measures thus allowed analysis to 
be conducted (as d is cussed in the next 



*An overview of "quantitative" versus ^'content" 
analysis is provided lor the non-technical reader. 

Quantitative ana :v sis , which composed the bulk of 
analysis porfornod on data in the present study, refers 
to the method of analysis whereby measures taken on 
variables are translated in'.o nathenatical language, i.e. 
numb ers , percentages, propositions, ratios, etc. 

Statistical methods are then employed as techniques 
to describe and evaluate the data. These methods enable 
one to state precisely --in mathematical language--the 
relationship bet ween variables. 

Not all data, however , are amenable to statistical 
analysis • Further, "statistical analysis" does not 
"tell the whole story" . In both instances, either as 
a substitute or a supp lement , a type of analysis called 
content analysis i-s performed. 

Content analysis , whereby verbal responses are sub- 
jected to ar.::lysi3, is an explanation of content for 
research purposes. That which nakes content analysis 
systematic and objective are the foil owing points: 

1. Categories--explicitly defined 

2. Methodical clnssifinntinn nf all material 

3. Some quantitative measurement employed 



chapter) to determine: 1) what "types" of 
reconraendat ions were :nade by consultants; 
and 2) what "types" of reconmendat ions 
were most f re quent ly imp lejaent ed by hos- 
pitals and what "degree" of iwp lenient at ion 
occurred for various "types" of reconinen- 
dat ions • 



Processing of Independent Measures 

All heretofore discussed data transformations have 
dealt with the dependent measures gathered in the 
present investigation. The reader will recall that 
various independent measures, i.e., hospital and medi- 
cal staff characteristics, consultant and interviewer 
data, were also collected. 

Raw data for independent variables were in one of 
three forms: 

1. Purely discrete data 

a . Dichot omous categories^-possibility 
for responses to fall into only one of 
two categories, e.g. , either yes c.c no, 
either full-time or part-time 

b. Trichotomous , tet rachotomous , etc. --possi- 
bility for responses to fall into only one 
of three, four, etc. categories, e.g. 

Degree possessed by hospital administrator 

1. Degree in Hospital Administration 

2 . M . D . 

3. R.N. 

4. No degree 

\ariable could occupy only one of the above 
four groupings 

2. Frequency data 

Frequency data answers the question "How many?" 



Essentially it is counting data, e.g., number 
of beds in the hospital. Unlike purely discrete 
data, frequency data can be ordered. For 
example, a hospital with 20 beds is said to 
have more beds tlian a hospital with 10. Often- 
times such data is treated as continuous data 
for purposesof analysis. 

3. Continuous data 

Unlike the above two cat egori za ^ ons of data, 
which can fall at separate point: on a scale, 
e.g., 0 , 1 , 2 , 3 , etc . , continuous variab les 
are able to occupy an infinite number of in- 
between values. For example, in the present 
study, the variables of "age of administrator" 
is a continuous variable, that is, it is possibl 
for the administrator to be 40, 40^4, ^Oh, etc. 

In the case of the independent variables, transfor- 
mation of the data was synonymous with descriptive 
analysis, and as a result the bulk of it will be dis- 
cussed in the following chapter. 



RESULTS 

The objective of the present section is to des- 
cribe aspects of the analysis process and to present 
relevant findings. Most of the data and findings will 
be summarized and reported in quantitative form, 
which presupposes some neasurement , and according ly, 
requires some nathenatical manipulation of the data. 
Quantitative expressions will, in most instances, be 
accompanied by qua]:>tative', interpretative, and explan- 
atory comments . In sone instances where data was not 
amenable to quantitative ana lys is , qualitative inter- 
pretations will compose the bulk of data analysis and 
interpretation. Both descriptive and inferential data 
analysis* of the dependent and independent measures 



*For the non- technical reader, a brief definition 
of these terms is provided: 



(continued on next page) 



will be presented. 

ANALYSIS OF DHPHKDIZNT MEASURES 



Hospital Change >!easures 
(gathered ir. both experimental and control) 



In-order to describe** what the groups were doing, 
means (X's), that is, average nuiiiber of changes and 
their standard deviations^, that is how widely indi- 
viduals vary from group X, were calculated for each 
group of the hospitals--experiinental and control. 
The results were yielded as shown in Table 7. 



* (cont inued) 

1. Descriptive statistics 

Descrxptive statistics embody an assessment 
and characterization of the group or sample. 
In giving an adequate description of the data 
one usually character! zes what is "typical'* 
of the group and indicates how widely indi- 
viduals in the group vary, i.e., aie they 
similar or different with respect to each other. 

2. Inferential statistics 

Inferential statistics make deductions or con- 
clusions regarding differences in data based 
upon statistical tests of significance. The 
process of drawing inferences, ma king pre- 
dictions, and testing significance are exampl es 
of sampling or inferential statistics. 



♦♦Descriptive statistics were in some instances 
utilized for their own worth, but in the bulk of cases 
used in order to conduct inferential statistics. 



Table 7 



X's and S.D.'s for Number of Changes Occurring in 
Experinental versus Control Hospitals (Changes 
Distributed According to Total, Educational, and 

•Non-Educational ) 



•Change lata 




Total 


Educa clonal 


Non-Educational 




X S.D. 


X 


S.D. 


X S.D. 


Experinental 


23.42 9.13 


8.72 


4.32 


14.69 6.89 


(N=36) 










Control 


17.86 9.20 


5.57 


4.29 


12.30 7.21 


(N=37) 











Inferential analysis (t tests of significance 
between mean number o if hospital changes) conducted on 
the data yielded the following findings (Table 8) . 
When total changes were considered, a statistical ly 
significant greater number of changes were found to 
have occurred at experimental hospitals than at con- 
trols, (t ^ 2.55; for 71 degree of freedom, significant 
beyond the .05 level of significance.) When looked at 
in terms of educational and no'^-educat ional breakdown 
of data, the following results emerged. Analysis of 
educational changes resulted in a t = 2.11, which for 
71 degrees of freedom is significant beyond the .05 
level of significance, the experimental hospitals having 
a significantly higher mean number of educational 
changes. Analysis of non- educational changes showed 
no statistically significant difference (t - 1.42). 
However, visual inspection of Table 8 permits one to 
view the occurrence of a higher (although not reaching 
statistical significance) mean number of non-educational 
changes occurring in experimental hospitals. This data 
suggests that both groups of hospitals had similar 
capacity for change. 



Table 8 



Phase I 



COMPARISON OF RIirORTHD C1IANGF.S: 
OXPliRIMONTAL f, CONTROL HOSPITALS 





TOTAL CHANGES 


Oxp . 
Cont . 


X S.D. 
23.42 9.13 

2.SS» 

17.86 9.20 




liD. C1!AN(;LS 


NON-HD. CliANGHS 


Exp. 
Cont . 


X S.D. t 
8,72 4.32 

2. 11« 

5.57 4.29 


X S.D. t 
14.69 6.89 

1 .42 

12.30 7.21 



• = p <.05 



Recommendation Inplencntation Measures (gathered in 
Experimental Hospitals only) 



Descriptive analysis was done on recommend at ion 
data in order to determine: 

1. What was the nature and extent of recommenda- 
tions transmitted by consultants to hospitals, 
and 

2. What was the nature and extent of recommenda- 
tions implemented by hospitals. 

A total number of 443 recommendations was received 
by 37 experimental hospitals, 353 in written form and 
90 made orally by the consultants (Table 9). Eighty- 
eight percent of the recommendations were educational 



in nature. The mean number of recoranendations Cwritten 
plus oral) per hospital was 11.97 with a standard 
deviation oft4.21. Individual hospitals received any- 
where fron one to 23 r ecoinracrdat ions . While oral rccom- 
dations r>layed an important part in the consultation 
effort, jocumcntation of written reconnendations was 
much more precise and consequently assessment of their 
outcomes nore reliable. Results reported in the present 
section will be limited to analysis of written recom- 
mendations . 



Table 9 

Nature and Frequency of Recommendations Transmitted 
to Experimental Hospitals 



EXP. HOSPITALS j 40 


HOSPITALS RECEIVING RECS. | 


57 


total 1 

Jo^PcM^J^'y^'' written 
TRANSMITTED 

oral 


445 


353 


90 


mean 

TOTAL NUMBER _ 
RECS. /HOSPITAL 

ransc 


11 .97 


4.21 


1-23 


EDUCATIONAL RECS. j 388 (SS?0 



An obvious feature of the 353 rccomjucndat ions 
transmitted was their heterogeneity. For purposes of 
describing the nature and variety of these recommenda- 
tions, a content analysis classification scheme was 
devised and each reconnendat ion was assigned to one 
of the categories. At the most general level, the 
categories were divided into educational and non-educa- 
tional recommendations. Educational included any recom- 



Aendation which had a primary objective of affecting 
the education of hospital personnel, e.g., ''iraplejacnt 
grand rounds," "initiate a team approach to education 
programs centered on the coronary care unit," "medical 
staff and administration jointly appoint a DME . " Any 
recommendation that did not specify a primary educational 
objective or was not directly related to the edu'" at ion 
of hospital personnel was placed in the non-education 
category, e.g., "develop an ICU," "provide salaried 
emergency ward staff coverage," "investigate the pos- 
sibility of providing home care services." 

Educational recommendations were further classified 
as relating to physician continuing education or relating 
to continuing education of nurses or allied health prac* 
titioncrs. Under physician education, three classes of 
recommendations were established: 1. resources for 
program development including personnel, facilities, 
materials and financial support; 2, planning for edu- 
cational programs including assessment of staff educa- 
tion needs, program desijjn and program scheduling; and 
5* program ir.pler.entat i on including institution of 
case-centered education activities, implementation of 
sub j ect -ori ented programs and development of house- 
officer training . 

On the other hand, content of non-educational 
recommendations was not further delineated by sub cate- 
gories due to the low frequency of these re commend at ions 
and their mi seel lane ous and discrete character. Table 
10 provides a content analysis of the recommendations 
indicating the numbers of recommendations within each 
classification and their implementation status. For a 
more detailed breakdwon r»£ the recommendations and 
their implementation status, refer to Appendix J ,pp . J2a*f . 

Of the 353 recommendat ions transmitted to the 
hospital, 31S (90-O were educational in nature, 296 (84^) 
specifically referring to physician education and 22 (6%) 
referring to nursing and allied healtii education. Since 
84 percent of the total number were subsumed under the 
general category of physician education, it was observed 
that consultants confi:i2d their efforts to the task of 
developing the medical education program in the community 
hospital . 

Further inspection of the 296 physician education 



Table 10 

Nu. ical Summary of the Implementation 
Status of General Catcijoriesoi* KecomT.endations 
at the End of 4 Months 



TOTAL NUMnCR OF ftCCOf.O^ENDAI tCNS 
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1 loq 
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33 


bO 


» 


f ifunc'jl Swppoii iw Trogt^ Dfvrlopm^t. fncluOinfl Hoip«U!. St^fl. antf 
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14 
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1 
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xecomnendations revealed that. J4Q (47^i> related to 
providing res ources for the physician education program. 
Resource-rclatcJ recommendations were equally represented 
in the following subcategories: hospital and staff 
financial support for progran developnent , educational 
materials and facilities, personnel sv.iport for the 
program, and inter-hospital sharing cf education re- 
sources. Each of these sub-categor ie averaged 24 
reconnendations. Forty-four recoanendations , however, 
were assigned to the sub-category of utilization of 
outside resources, Tiie occurrence of a higher frequency 
in this sub-cate^orv nay be accounted for by 16 recom- 
mendations calling for utilisation of extra-hospital 
teachers in the education progran. 

The second najor category under physician education 
reco:araendations, educational prograr. nlnnning , contained 
fewer reconnendations tnan the resource category. Of 
the 62 (21?*) reco:nnendations in this category, 32 (52-s) 
pertained to designing the progran such as stating pro- 
gran goals, striicturin-- the activities, assigning staff 
responsibility for inplenent ing the progran. The mechan- 
ical aspects of prograri, such as devising a tine sche- 
dule, publicizing the sessions" and keeping attendance 
records, received 25 (.>7?i) reconnendations. Recomnenda- 
tions pertinent to ass'.ssnent of staff educational needs 
and interests and progrr..n evaluation, a nore difficult 
task in progran planning, were nade in only 7 (11%) in- 
stances. This nay have been a reflection of the consul- 
tants' judgenents that nost connunity hospitals were not 
ready to take on something so rigorous at that time. 

The third najor category, progran im>lencn tat ion , 
received 94 (32'i) of the physician education reconnen- 
dations. Fifty-two (55%) of these reconnendations were 
in the area of oatient r-ntered education activities, 
such as instituting grand rounds, CPC's, or mortality 
conferences. The high frequency of this sub-category 
reflects PMI's philosophy that tho connunity hospital 
education progran should be based on the specific 
patient care probluns of that hospital and that such a 
nropram would be nore relevant to the education needs 
of the staff phvsicians. Eighteen (19%) reconnendations 
transmitted sJjecified inpl oncntat ion of sub j ect -oriented 
education activities, such as lectures, courses, and 
specialty-care education prograns. Eleven (12i) recom- 
mendations pertained to the development or improvement 



of house officer training programs. 



In the process of advising community hospitals as 
to devclopncnt of their physician education program, 
the consultants provided some ho Ip and reconmencat ions 
regarding continuing nurse and al lied health education 
programs when deemed appropriate . Twenty -two (7'0 of 
the total number of educational recommendations were 
made in this area of which 15 (S9-0 related to joint 
physician-nurse education activities, such as inviting 
nurses to phvsician educp^ion programs and organizing 
special joinr physician and nurse education sessions 
on the care of patients in such specialty-care units 
as the ICU and CCU. Other nursing recommendations per- 
tained to developing personnel resources for the program 
such as inviting visiting nurse consultants and appoint- 
ing directors of in-service teaching and establishing 
in-service nurse education programs. 

Non-educat ion recommendations numbered 55 (10% of 
total number of recommendp tions transmitted) . Little 
relationship existed between the various recommendations 
in this category other than they had little or no 
specified bearing on the development of education pro- 
grams. Examples of such recommendations are making 
vider use of emergency room, establishing specialty 
clinics, and establishing-.department chiefs. None of 
the non-educational subcategories contained more than 5 
recommendations . 

Regarding the overall fate of the 555 r ecos'.aendations 
transmitted by consultants to hospitals during Phase I, 
the f o 1 lowing f igures emerged : at the end of the four 
month implementation period, 130 (57%) of the recommen- 
dations were fully operational, 69 (20%) were partially 
implemented, and 151 (45%) were rejected. One hundred , 
ninety-nine (57%) of the recommendations had been 
accepted and iiad some^action taken on then (IPT rate). 

In reviewing the implementation rates within the 
various recommendation categories , it was found that 
the implementation rates were quite consistent with the 
overall implementation rate of all 555 recommendations. 
This held true at the level of educational and non-edu- 
cational r jconmendat ions , with the non -educationa 1 
having a slightly lower full plus partial implementation 
rate (IPI), that is, 49 percent as compared with a 57 



percent IPI rate for educational recommendations; at 
the physician education level CIPI=57^0 and the nursing 
and allied health education level (IPI^SD'l); and at 
the sub-category levels within physician education, in- 
cluding progr.n resources (IPI=56v) and program planning 
(IPI=52o). The program implementation level of physician 
education had a slightly higher IPI (65-0 • 

Only upon further inspection of more detailed 
classification were discr epencies in implementation 
rates observed, and even these were not significant. 
Lower IPI rates occurred in the sub categories of 
scheduling education activities (4Svi) and assessing 
education progrnns (43%). This latter figure is based 
upon only seven recommendations of which three were 
fully or partially implemented. Higher IPI rates 
occurred in the areas of instituting joint physician- 
nursing education programs (69%), developing house 
officer training (73%) and implementing case-oriented 
editcation activities (65'i) . Caution should be taken 
in interpreting these data, as the results are based 
on very low frequencies. 

The 10 most frequently made recommendations and 
their implementation status at the end of four mon.ths 
;&re listed in rank order in Table 11. The eleven types 
of recommendations listed account for over half of the 
total number of recommendations. This fact suggests 
that these recommendations may reflect some of the most 
comjaon educational needs in '.community hospitals, or it 
may indicate something about consultants* biases. 



ANALYSIS OF IXDEPE,\DENT MHASURES 

m 

The research question posed for purposes of the 
present study was of the nature: V/hat is the effect of 
X on Y; X being independent measures, Y being dependent 
measures. In testing the manipulated (treatment) inde- 
pendent variable, that of consultation, the question 
was of the nature: What is the effect of consultation 
on program development, i.e., change measures. The 
question was answered by determining nature and extent 
of the recommendations implemented by experimental 
hospitals, and by comparing consulted versus non-con- 



Table M 

Most Frequently Submitted Reconuocndat ion. 



Number 
Submitted 



31 Use guest educators 

22 Hospital provide education 
budget 

19 Establish case presentations 
17 Develop library 

15 Develop inter-hospital education 
program 

13 Establish/develop DME position 

11 Develop intern/resident program 

9 Use external resources for program 
deve lopnent 

9 Establish formal lecture series 

9 Rotate staff respor.sibility for 
planning/implementing program 

9 Restructure education program 



Number Fully 
Implemented 
after 4 Months 

11 
6 



9 
7 



6 
6 
2 

1 
3 



suited, or experimental versus control hospitals, on 
the hor ital change measures, as discussed previously. 

It will be recalled by the reader that a corollary 
research objective of the present study was to determine 
factors within the hospital v;hich nay affect (either 
facilitate or inhibit) the establishment and implemen- 
tation of a continuing education program. These factors 
were defined as fixed independent variables and are 



delineated in Appendix h-1 of the present report. In 
testing fixed independent Cclassification) variables, 
the research question became: What is the effect of 
hospital characteristics, nedical staff charact eri st i cs , 
consultation characteristics, etc. on impact of con- 
sultation.* These "fixed" variables are the focal 
point of the ensuing discussion. 

Transformation of "fixed" independent measures, as 
noted in the preceding chapter, was synonymous with 
descriptive analyses. Descriptive analyses for the 
independent* ncasures detail the type of data and form 
of data description yielded for each variable . Most 
of the independent measures were not described for 
their own worth, but merely ca tegori zed to accomodate 
infer jntial analysis. 

Since "type" of data yielded dictated method of 
inferential analyses performed, independent measures 
are also presented in terms of the nature of the data. 
Two major categories or "types" of data emerge:! 

1 • Categorical data --possibili ty for responses 
to fall within only 1 of 2 or more categories 

a. Either/or data (2 categories onl)) 

b. Mult icategorical data (3, 4, 5 categories) 

2. Continuous data --possibility for responses to 
occupy an inf ini te number of in-between values 

Variables were distributed into four categories: 
hospital characteristics, and medical staff characteris- 
tics, which will be discussed as a unit, followed by 
discussion of consultant characteristics, and interviewer- 
interviewee characteristics. 



Hospital and MedJca] Staff Characteristics 

A listing of the hospital and medical staff charac- 

*Dependent measures utilized to define impact of 
consultation were : 

1 . " I " and " I P I " ra t es of r ecommendat i ons 

2. Number of reported educational changes. 
Because of time limitations, due to the awarding of the 
extension ot the contract for two wore years (Phas4J II) 
and the subsequent amount of preparation needed to im- 
plement Phase II, it was not possible to conduct analyses 
of indeuendcnt variables with the dependent measure of 
number of reported educational changes. 



tci'-istics used in analysis arc presented in Table 12. 
Data for various additional characteristics, as listed 
on the "Hospital Attribute'' form (see Appendix h-1) was 
collected al though not analy-cd . In such instances 
either 1) inadequacy of data (invalid, unreliable, or 
insufficient), or '2) insufficient time, precluded a 
more complete analysis. 

Column 1 of Table 12 details the Tormat of data 
description for each variable. Distribution of data 
into catef^ories is presented in column 2 of Table 12 
Respective methods of analysis used are presented in 
Column 3, and results in Column 4. Analysis of results 
will be discussed further in the present section. 



Consultant Characteristics 

As discussed under the data collection section of 
the present report (pu.i2 - 53) inclusion of consultant 
and interview characteristics in the measuring process 
servel as a means of taking cure of possible uncontrolled- 
for variables arising from unwanted sources. 

Although some of the various consultant character- 
istics were originally solicited from the hospital, 
more reliable data was gathered from PMI records for 
various characteristics. As a result, the following 
characteristics* associated with the entire consul ta- 
tion process emerged (Table 12). 

Unlike the hospital and medical staff characteris- 
tics which a^re, for the most part, self-explanatory in 
nature , thg consultant and interviev;er characteristics 
presented in Table 12 require operational definitions, 
whi ch are presented bel ow . 



*A few are taken directly from the "Hospital Attri- 
bute" form (Appendix H-1) wany are Kiodific4- or further 
explicated versions of those appearing on the form, 
while still others are new va- .ables acquired by refer- 
ence to PMI records. 



Table 12 
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Relationship of Hospital, Medical Staff and 
ConsuYta^on Characteristics to Full I-Plenentation ("I") 

of Reconnendations 



Fortaat of 
Data 

Description 



Nature ("type") Method of 
of Data Analysis 



Result 



"I" "IPI" 



\ 

J 



J 



0 



r 

L 



L 

[ 



HOSPITAL ATTRIBUTES 
Eedsize 



Sphere of influence 
(proximity to 
medical school) 

States in which 
hospitals are 
located 

Administrators 
holding degree in 
Hosp* Adm. 

Type of control of 
hospital 



left in form 
of counting 
data 

left in form 
of counting 
data 

Maine vs* 
Mass* vs* 
N*H* vs. R.I* 



ICU facility! 

Budget for 
education 

DME 



yes/no 



voluntary 
non-profit 
MS. "Other" 

"yes/no 

yes/no 



yes/no 



Education committee yes/no 

Resident and ycs/no 
Intern program 

Informal nursing yes/no 
education program 

External nedical yes/no 
Audit 



continuous 
continuous 



multi- 
categorical 



Pearson r 



♦.03 



Pearson r -.001 ♦•IS 



•SS 1-62 



categorical 


t 


test 


1.41 


2.8J* 


categorical 


t 


test 


1.25 


1.10 


categorical 


t 


test 


.09 


1.35 


categorical 


t 


test 


.89 


.90 


categorical 


t 


test 


.08 


.22 


categorical 


t 


test 


.47 


.69 


categorical 


t 


test 


.75 


1.30 


categorical 


t 


test 


.95 


±± 


categorical 


t 


test 


1.93 


2.07 



tt Data was insufficient to perform computation. • . 



i; 
\t 



IL 



Continuation of Tabfc 12 

Relationship of Hospital. Medical Staff and 
consultation Characteristics to Full J-P l^^J^^f^^^^ ^"^"^ 
and Full plus Partial Implcncntation C'lPI ) 
^ of Reconncnuations 




II. MEDICAL STAFF 
ATTRIBUTES 

Ratio of active 
staff to number 
;Of occupied beds 

% specialists 
to G*P* 's 



% specialists 
board certified 



X age of staff 



% active staff 
with other hosp* 
affiliations 



left in form 
of* counting 

left in foria 
of counting 
data 

ieft in form 
of counting 
data 

left in form 
of counting 
data 

left in form 
of counting 
data 



II. CONSULTANT 

CHARACTERISTICS 

PHI experience of 
consultant 

Attendance at 

consultant 

orientation 

Number of visits 
made by consul- 
tant to hospital 

Transmittal of 
recommendations 
on time 

Availability ^of 
data from PMI 
forms used by 
consultant 



yes/no 
yes/no 



1 visit vs 

2 or more 
visits 



yes/no 



low (0-3 
forms avail • 
vs. high 
(3^5) 



continuous 



continuous 



continuous 



continuous 



continuous 



categorical 



categorical 



categorical 



categorical 



categorical 



Pearson r -%18 



Pearson r .08 



Pearson r 



1.34 
1.88 



• 88 



% 



19 



Pearson r .18 .19 



-.30" -.36* 



Pearson r -.20 -.05 



.12. 
1.79 



.31 



1.70 1.27 



P^.OS 



)inputation . 



Tabic 12a 
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Relationship of Intcrvicwer^Intcrviewcc 
Characteristics to Number of Reported Changes 
at Both Experinental fi Control Hospitals 



Format of 
Data 

Description 



Nature ("type") Methods of 
of Data Analysis 



Result 



IV. INTERVIEWER- INTERVIEWEE 
CHARACTERISTICS 



Number of Inter- 
viewees 



Quantity of data 
collected by Inter- 
viewers (number of 
hospital changes) 



<3 vs. 3 or 

more indi- 
viduals in- 
terviewed 



categorical 



Interviewer A mult icategorical F 
vs. B vs. C 



Inteiviewer D mult icategorical F 
vs. E vs. F 



Cont 
Exp. 



±± Data was insuffTcient to perform computation. 
*• P^OS 
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X . P>1I experience of consultant 

Divided into two groups: prior experience 
versus no prior experience, prior experience 
being defined as working in some capac i ty 
with PMI and as a result being familiar with 
its philosophy and forms (instruments) . (See 
Appendix A for review of Stearns's article, 
"Positive Approaches to Continuing lledical 
Education" . 

2 . Attendance at consultant orientation 

Divided into two groups : attenders versus 
non-at tenders in reference to initial con- 
sultant orientation session (June, 1967) 

3 . Number of visits made by consultant to hospitals 

Divided into two groups on basis of visits: 
one visit versus two or more visits 

4 . Transmittal of recommendations on time 

As previously noted, recommendations were to 
be transmitted to the hospital by November 1, 
1967. Hospitals were categorized into two 
groups on the basis of whether recommendations 
were transmitted to them on or around November 
1, 1967 versus those hospitals whose recommen- 
dations were transmitted to them well after 
November 1 , 1967 . 

5 . Availability cf data from PMI forms for use 
by the consultant 

Concerned with farms to be filled out by the 
hospital and to be made available to the con- 
sultant for his use while devising recommen- 
dations (discussed under "Data Collection for 
Purposes of Program Development," pp. 28-30) 
Cat eg or i zed into two groups : 

a. Low: 0-3 forms available to the consul- 
tant , versus 

b. . High: 3-S forms available to the consul- 

tant 



Interviewer - 



Interviewee Characteristics 



1. Numb er of int erv ievee s (people interviewed in 

the hospital) 

Data was categorized into two groups as follows: 

a. Less than 3 interviewees, versus 

b. 3 or more interviewees 

Separate analyses were conducted on experimental 
and control groups • 

2 . Quantity of data collected by interviewers 

Hospitals were categorized into five groups 
according to the interviewer who visited the 
hospital. Analyses were conducted in order to 
determine if a significantly larger degree of 
variance occurred in the number ofreported 
changes as a function of the interviewer who 
was collecting the data. 



Categorical Data 

With reference to Tables 12 and 12a (pp.61 - 63), all 
"categorical" data described as either/or (yes/no, or 
reduced to two cat^; -^ories) underwent a test of signifi- 
cance between mean X) implementation rates ("I*' and 
•'IP-.") for the two groups (e.g.*, yes vs. no). Signifi- 
cantly yielded findings are presented below. 

Statistical Significance (defined as .05 or less level 
of probability*y 



♦Probability of 5 chances or less in 100 that 
observed differences are due to chance factors. 



1. External medical audit and IPX rate 



Those hospitals possessing an external medical 
audit appeared to have a significantly lower 
IPI rate (t=2,07, p<,OS), t IPI with external 
medical audit was 41,4% (N=ll); X IPI without 
external medical audit was 64.8% (N«26) • The 
present finding may categorize the external 
medical audit as an inhibiting factor, 

2 . Degree in hospital administration and IPI rate 

Those hospitals in which the administrator 
possessed a degree in Hospital Administration 
yielded a^signif icant ly lower IPI rate (t=2,83, 
p<01). X IPI with a degree was 37*6% (N=10) ; 
X IPI without a degree was 6S,4% (N«27) • The 
above finding seems to indicate thut possession 
of an MHA may be an inhibiting faci:cr. 

Trend Toward Significance 

Although not reaching statistical significance, a trend, 
both "strong" and "moderate" in that direction was dis- 
cernable for a number of variables. "Strong" is defined 
as psClO. "Moderate" indicates ps=<#30 or less. Pr'obabi- 
lities above*30 were considered negligible, and the 
variable was discounted as a possible facilitating or 
inhibiting factor". (The above definitions of "strong" 
and "moderate" will apply to all analyses discussed in 
the present chapter) . 

Strong Trends 

1 . Existence of external medical audit and I rate . 

Those hospitals utilizing an external «;edical 
audit system yielded a lower full implementation 
rate of recommendation (t = 1.93, p<^10) T I with 
a msdical audit was 26.2% (N=ll); X I without 
a medical audit was 43.4% (N^26) . 
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2. Availability of data from PMI forms for use by 
consultants and I rate > 

Those hospitals who completed 3-5 PMI data 
collection forms for use by the consultant 
had a higher fiill implementation rate. (t^ 
1.70; pL<.10) X I with high frequency of 
forms was 49.3% (N«12) ; X I with low frequency 
of forms was 33% (N»25). 

3. Attendance at consultation orientation and 
both I and IPI rates. 

Those hospitals whose consultants attended 



1 . Existence of an Intensiv* Care Unit (ICU) 

Those hospitals who possessed an ICU unit 
had a higher full plus partial implementation 
rate. (t=l-33, p^.25). X IPI with ICU was 
62.6% (N=18); I IPI without ICU was 53.9% 
(N-19) . 

2 . Existence of a residency and intern program 
and IPI rate. 

Those hospitals who had an intern and resident 
program had a higher full plus partial im- 
plementation rate. (t = 1.30; p^.25) Ic IPI 
with program was 73.8% (N=6) ; Y IPI without 
program was 54.8% (N»31) . 

3 . Possession by administration of a formal 
derree in hospital administration and I rate. 

Those hospitals who had an administrator with 
a degree in hospital administration had a lower 
^ull implementation rate. (t=1.41, p^.25) 
X I with degree was 29.4% (N«10);T I without 
degree was 41.5% (N=27) . 




Moderate Trends 
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4 ' control C/f hospital and I rate. 

Other" hospitals, e.g., church operated, 
profit, yielded higher full implementation 

than voluntary non-profit. (t»1.25, p 
" I rate for voluntarv non-profit 
ho s was SA.y\ (N-29); X I rate for 

"ot. hospitals was SO. 4% (N«8) . 

5 . Speed of response to PMI invitation and 
I rt^te. 

Those hospitals responding within 0-3 weeks 
yielded a higher full implementation rate 
than those hospitals who responded within 3 
or more weeks. (t«1.28, p .30). ^ I rate 
for early responders was 41.9% (N«28) ; X I 
rate for late responders was 27.1% (N»9) . 

6. A vailability of data from PMI forms used by 
consultant and PMI 

Those hospitals who provided their consultants 
with more PMI data collection forms had a 
higher full plus partial implementation rate 
(t«1.70, P4C30). T IPI rate of those hospitals 
providing more forms was 4. 3% (Nal2); X IPI 
rate of those providing f' .cr forms was 33% 
(N-2S) . 



Multi categorical Data 

The following chc ractcristics classified as multi 
categorical data» as depicted in Tables 12 and 12a, 
underwent analysis of variance (F test^) . 



1. States 


in which 


hospitals were located 




X I rate 


F test 


X IPI rate F-test 


Maine 


41,2 




71.3 


Mass . 


41.6 




60.3 






.546 


1.C2 


N.H. 


37.6 




46.1 


R.I. 


22.0 




42.8 
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Analysis of variance (F tests) conducted on the 
above data yielded findings not reaching statistical 
significance; therefore there are no apparent differences 
in X implementation rates as a function of the state in 
which the hospital was located. 

Visual inspection of the above data, however, will 
permit the reader to view that for IPI rate an F of 1.62, 
denoting a p=<.10 indicates a strong trend toward sig- 
nificance. (Massachusetts and Maine yielded higher 
full plus partial implementation rates.) 

2 . Quantity of Data Collected by Individual Interviewers 

Analyses of the number of reported hospital 
changes as collected by individual interviewers 
yielded results indicating that there was a 
statistically significant difference among 
interviewers in terms of the amount of data 
they collected. (F=8.34, P<.01 for group of 
3 interviewers who visited a^fewer number of 
hospitals) and (F=3.54, p<.b5 for group of 
3 interviewers who visited a greater number 
of hospitals) 

The observed interviexver bias, it could b-e 
concluded , contaminated the findings regarding 
the number of hospital changes reported by 
experimental and control hospitals . However, 
since each interviewer's hospitals included 
approximately the same number of experimental 
and control, it can be assumed that the ob- 
served bias was distributed somewhat equally 
between experimental and control hospitals 
and that the findings were not invalidated. 

Continuous Data 

All data discussed below, continuous in nature, 
were subjected to Pearson r correlations. The purpose 
of analysis was to show how and _to what extent con- 
cepts of change and independent variables were inter- 
related. Hospital and medical staff characteristics 
embody the independent variable; dependent measures 
utilized were I § IPI rates. Discussion of the following 
results should be read with reference to Tables 12 g 12a. 
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Ho J pita 1 Characteristics 

1 . Bedsize 

No St «A.cist ically significant correlations wei e 
yieUed for bedsize and full implementation rates 
of recommendations. (r=+.03, N=37) or '*>r bed- 
size and IPI rates (r==-.ll, N=37) 

2 . Proxinity to Medical School 

Although no statistically significant correlations 
were yielded , an apparently moderate trend 
(r-+.18, N=37 p<.10) was noted between medi.al 
school influence and IPI. Hospitals located 
a greater distance from a medical school in- 
fluence appeared to implement (and/or partially) 
implement proportionally more of their recommen- 
dations . 

3. Ratio of active staff to occupied beds: 

With the purpose of reflecting the* "busyness" 
of the medical staff, a correlational analysis 
was undertaken on the above ratio and the 
implementation rates of recommendations, al- 
though no statistically significant relation- 
ships were yielded, a moderate trend towards 
significance appeared for both I § IPI rates, 
(r ^=-.183, p-c.lO, and r ==+.192, p^T.lO, 
N-37 in both analyses]^ I rates produced a 
trend towards positive correlation (i.e. , 
larger the ratio of staff to occupied beds, 
the greater the I rate) while IPI rates re- 
sulted in a trend towards a negative correla- 
tion (i.e., lower the ratio, reflecting a less 
busy staff, the higher the IPI rate) 

4 . Ratio of active staff to number of beds 

Negligible correlation ratios were yielded 
(r :=+.01 and r:=+.02) . 
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5 . Mean age of active medical staff 

A statistically significant negative correlation 
(r=-.35, p<.05) was yielded for x age of staff 
and IPI rate. Also, a strong negative trend 
(r=-.29, p=less than . iO) was noted for above 
variable and I rate. The findings seem to 
suggest that hospitals \cith a lower mean age 
for the medical staff implement more of their 
recommendations • 

6- Percent Specialists to G.P/s 

Above ratio reflected composition of medical 
staff. Higher ratio indicated a greater % 
of specialists, while a low ratio reflected 
greater # of G.P.'s. No statistical ly signi- 
ficant results were yielded, although moderate 

i^ere discernable. For^both I and IPX 
rates moderate trends (r =+.178, r=+.l8S, 
p<.20) towards a positive correlation appeared, 
indicating that hospitals with a higher con- 
centration (percent) of specialists on the 
staff tend to demonstrate a trend towards im- 
plementing more recommendations. 

7. Per cent Specialists Board Certified 

Negligible correlation ratios wero yielded. 

8- Percent of staff with other hospital affiliations 

No statistically significant findings were 
yielded, however, a somewhat moderate trend 
towards a significant negative correlation 
(r=.20, p=^.10) resulted for I rate implying 
the fewer staff members with other hospital 
affiliations, the higher the number of 
recommendations ful ly implemented . 



DISCUSSION OF RESULTS 



Although preliminary and somewhat tentative, results 
of Phase I research do suggest that: 

1. The utilization of an education consultation 
procedure and process does in fact have a sig- 
nificant impact on the development of con- 
tinuing physician education in local community 
hospitals, particularly when short-term 
selective indicators of perceived change are 
utilized . 

2. The data strongly support the necessl-y of 
prior training of the physician consultant if 
one is to maximize his impact on the hospital. 

3. The greater the systematic effort on the part 
of the medical staff of the hospital in pre- 
paring its educational programs (e.g. the use 
of educational activity sheets and medical 
staff questionnaires), the greater the like- 
lihood of innovation and change in educational 
activities . 

In short, educationally trained physician conrul- 
tants , involving community hospital physicians in r. 
systematic effort, do bring about changes in continuing 
education for physicians. It also follows, though not 
as systematically documented in the present research, 
that a variety of administrative and organizational 
characteristics of the institution will tend to facili- 
tate the rates of innovation and change of educational 
programming of physicians . 

The data at hand are not without some promise for 
suggesting a feasible strategy for extending education 
consultation to community hospitals. That difficulties, 
still exist in both the intervention and evaluation of 
such efforts at altering and improving educational 
programs for hospital based physicians goes without 
saying. In the present instance it may be instructive 
to briefly characterize some 6f the more explicit 
difficulties experienced. In the realm of the con- 
sultation (intervention) procedures, three significant 
problem areas arose. 



The first relates to the issue of the recruitment 
and training of the medical consultants to the community 
hospitals. Since all of the consultants were pursuing 
this specific activity as a part time task in addition 
to full time commitments at teaching hospitals (in- 
volving programs or service teaching and research), 
instances of conflict and delimitation of commitment to 
the consultation requirements were manifested by various 
members of the education consul tan t team . The behavioral 
results were instances of cancellation of meetings with 
representatives from the experimental hospitals, re- 
striction of the duration of consultations, sporadic 
participation in pre-consultation orientation activities, 
and extreme variation in the delays in the completion 
of field reports and transmission of recommendations 
to experimental hospitals. It appears that the hospitals 
of those consultants who attended orientation session 
implemented more of their recommendations. This finding 
suppcTts the contention that training of consultants 
produces positive impact on the consultant's effective- 
ness and further on educatio.ial program development. 
As a facilitating factor this finding, coupled with 
consultants' expressions of feelings of inadequacy in 
providing consultation, demonstrates support for and 
value of the proposed systematic training program under- 
taken during Phase II. 

The second major difficult) in the consultation 
process was the highly variable utilization of various 
survey forms (i.e. the educational acti*^ ities formis 
and the staff physician questionnaires) by both the 
meaXcal consultants and the representatives from the 
experimental hospitals. The resistance to the time 
and energy required to. collect data of pertinence to 
educational progranming was frequently an issue in the 
consultation and a correlating condition in the ensuing 
rate of change and innov- ion in the experimental 
hospitals. 

Thirdly, the impact of consultation on development 
of physician education programs was found to be highly 
dependent upon certain intra -hospital circumstances 
and schedules relating to budgeting, staff procurement, 
etc. which in many instances appeared to block inno- 
vation and change. The primary implication here is 
the necessity for lead time in the planning and scheduling 
of intervention with other on-going decision making 
and scheduling activities (particularly financial) 



which operate in detexjnining the likelihood of innova- 
tion and change. 

Regarding the difficulties encountered in the 
evaluation of such an educational intervention, the 
following instances are worthy of report. The single 
most salient difficulty encountered in the evaluation 
research described here related to the matter of ob- 
taining data concerning the on -going educational 
activities at the experimental hospitals, prior to 
and during the consultation process. Specifically 
t\io distinguishable issues are involved, the first 
relates to the general inadequacy of records pertaining 
to activities of the physician staff within the in- 
stitution (i.e., hospital recordc are generally ex- 
tremely inadequate regarding the content , part ic ipat ion , 
and the nature of out comes of educational activities.) 
The second issue, reflects a lack of commitment to 
generate and/or a failure to appreciate the potential 
utility of such information, including data on the 
hospital staffs* needs and/or willingness to participate 
as teachers or students in learning activities, in the 
continuous planning and implementation of educational 
programming by members of the hospital's medical staff 
and administration. In the case of the current study, 
f ai lure to obtain the inf ormat ion , not only had im- 
plications for the success of innovation and change in 
educational programs, bu also markedly delimited the 
variety of analyses to issess the effectiveness of the 
consultation process e.g. failure to obtain reliable 
indicators of pre- consultation educational activities 
prevented the making of comparisons with post-consul- 
tation educational activities to assess significance 
of change. Similarly failure to obtain more systematic 
data on the attributes of physician staff prevented 
the mor profound examination of the relationship of 
the-medical staff attributes which might relate to 
increase in rate of educational innovation. 

In terms of the ultimate need for evidence of the 
efficacy of various forms of intervention on the long 
term change in development of continuing physician 
education prog JiS , it is concluded th^,t the study 
during the first year (Phase I) did not use a particularly 
optimal interval of implementation (i.e. four months) 
in which to assess the degree or rate of implementation 
and change. This conclusion is based on the assumption 
that the time neededto fully affect change in such a 



complex system as a hospital may exceed the 4 month 
implementation period that was allotted for evaluation. 
Furthermore, no opportunity was provided to assess 
degree of maintenance or continuation of innovation 
and change. The awarding of the continuation of the 
study for two additional years (Phase II) provided 
some opportunities for additional and delayed measures 
of both implementation and maintenance of innovation 
of hospital-based physician education programs. 




PHASE II 



STUDY OBJECTIVES 

On April 1, 1968 the extension of Contract No. PH 
108-67'"170* for two additional years was awarded by the 
Bureau of Health Manpower** to be effective May 27, 1968. 
Two years were allotted in order to provide another set 
of consultations during the first year and to allow time 
for post consultation data collection, analyses and final 
report writing during the second year. Several factors 
contributed to the granting of the extension (snbseniient ly 
referred to as Phase II). First, Phase I initial findings 
provided preliminary support for the efficacy of the 
consultation, and thereby indicated that the consultation 
warranted further development and study. Second, since 
consultation appeared to be an effective stimulant to 
CME program development, the next logical step was to 
determine the relative impact of various amounts of 
consultation. To pursue this question Phase II was to 
incorporate two levels of consultation , "intensive " con- 
sultation and "minimal" consultation, which would 
operationally be defined by the numbers of visits hos- 
pitals would receive from consultants. Third, results 
of analyses, our own observations, and the consultants 
and local education coordinators statements of felt needs 
for more information and greater orientation to CME pro- 
gram development indicated the need for more formal train- 
ing as a supplement to consultation. Development and 
evaluation of such a training program would be a primary 
focus of Phase II. And finally, in terms of evaluation, 
obtaining a true indicator of the impact of consultation 
was limited by the time factor of Phase I*s one year 
study. The recommendation implementation period al lotted 
during Phase I was too short (4 months) to allow hospitals 
to fully implement many of the consultants recommendations 
and occurred at a less than optimal time of the year 
(November-February) for acting on recommendations. (See 
Phase I - Dicussion of Results for more detailed dis- 
cussion of factors leading to the initiation of Phase II) 

Phase II objectives are concisely summarized in the 
Scope of Work provided by the contracting agency. They 
are as follows: 

* Current number is NIH 70-4150 

** Current title is Bureau of Health Professions Education 
and Manpower Training 
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I 

I 

1. Utilizing experiences and procedures developed 
in Phase I , continue consultation services to 
selected community hospitals in the four state 
region. The consultation service shall be 
varied so that half of the hospitals receive 
five visits in a 13 months' period (intensive 
consultation) and half only two visits in that 
same 13 month period (minimal consultation) • 

2. Continue and conclude the evaluation of the 
effectiveness of the consultation method In 
estaolishing and maintaining educational 
activities and the selected community hospitals. 
Specifically, such evaluation shall include: 

a. The extent to which the recommendations 
of the contractor are impl ement ed at 
the community hospital . 

b. An assessment of the relative efficacy 
of intensive vs. minimal consultation 
(as defined in 1. above) in the imple- 
mentation of the contractor's recommenda- 
tions . 

3. Continue to- describe the factors, both in the 
consultative service and in the hospital, which 
materially affect extended maintenance of phy-. 
sic i an education programs . 

4. Design and execute a training program for phy- 
sicians who will provide consultation services 
to community hospitals in the region. Th'e pro- 
gram shall be developed along sound principles 
of educational methodology. Curriculum shall 
include units in educational methods and techno- 
logy > behavioral psychology , avai lab le educat iona 1 
resources, utilization of health manpower, and 
others . Approximately ten consultant trainees 
shall receive the equivalent of seven (7) man- 
days of training . 

5. Design and execute a training program for phy- 
sicians who will serve as local education co- 
ordinators or directors of medical education. 
The objective of this program is to make local 
coordinators better able to cope with problems 
of continuing medical education on a sustaining 



1 



basis. A local coordinator or director of medical 
education from each of the 40 participating 
hospitals shall take part in six (6) man-days 
training , 

To summarize the research component of the study, 
the primary objective of the impact and effect of the 
consultation process on continuing medical education pro- 
gram development and implementation, while the corollary 
research objective was the determination of factors 
within the hospitals and consultation process which 
facilitate or inJiibit implementation and establishment 
of cont inu^iag -i^ducati on— pi^-o^rams for phys ici ans • 

Phase II Scope of Work in relation to Phase I can be 
found in Appendix B. 



STLi, DESIGN 

Phase II project design was to continue implementa- 
tion of Phase I*s original design (pp . 8-10 )with the 
exception of some modifications and supplements. 

Consultation 

Consultation service extended to experimental hos- 
pital was to be dichotomized into minimal and intens ive , 
that is, half the hospitals were to receive two visits 
and half five visits respectively. This was done in an 
attempt to further the nominal either/or analysis of 
consultation vs. no consultation into an analysis of the 
extent of c or, suit at ion as it relates to changes in CME 
program deve lopment . 

Training 

The most important aspect of Phase II was to be the 
modification, and complete reworking and improvement of 
previous training held for both consultant and interviewers 
during Phase I anJ the addition of a new trainin^^ program 
for hospital phys ician education coordinators. Training 
for consultants and local education coordinators was to 
be translated into 6 and 5 days respectively of training 
for each group. Discussion of the purposes, procedures 
and outcomes of the training programs can be found on 
pp. 85-100. 



Research 



Pertinent to the original pre -post experimental 
group design (pp. 8-9), comparing quantity and nature of 
CME activities in experimental hospitals before and 
after consultation, which was not executed due to failure 
in collecting the necessary pre da ta , Phase II research 
design called for reenactment of the design by resoliciting 
both pre and post data. Local physician education co- 
ordinators would be asked to complete an education acti- 
vities sheet for each year of the study - one depicting 
nature and extent of activities prior to consultation, 
one reflecting activities at the^^end of the first year 
of consultation and another summarizing activities after 
the second year of consultation. Analysis would consist 
of determining significant changes in program development 
activities as a result of consul tat ion intervention. 

Consultants recommendations would be followed up 
and analyzed as in Phase I*s design but with one major 
variation. Instead of the original 4 month recommendation 
implementation period of the first year, a longer 8 month 
period would be a^llotted - a more realistic time period 
for hospitals to take action on recommendations. Also 
recommendations were to be submitted to hospitals by the 
middle of September* which would more closely coincide 
with the time per: in which the yearly CME program is 
initiated. To det^xmine the effect of consultation on 
program development over an extensive period of time. 
Phase I recommendations would be fol lowed up 18 months 
after they were submitted. 

The second aspect of the original design, the con- 
trol vs. experimental group design (pp. 9-10) would remain 
in tact. However, attempts woul d be made to include 
two additional components to the assessment of reported 
changes occurring at experimental and control hospitals 
by 1) examining the level -of completion of a change, 
e.g., a fully operational change versus one that had 
only reached the planning stages; and 2) conducting a 
more detc ied content analysis of reported changes. 

The corollary research objective of uncovering 
factors in the hospital, the medical staff and the con- 
sultation process which may effect innovation and change 
in CME program development would follow the same design 
as Phase I (p. 11). Attributes would be reviewed for 



possible deletion from study as a result of the inability 
to obtain measurable data or the remoteness of relation- 
ship of the hypothesized variable to the dependent variables. 
Attributes not included in Phase I would be considered for 
inclusion in the study if they were feLt to be a possi- 
ble contributing factor and if they were measurable. 

All research instruments and procedures were to be 
reviewed for possible refinement in an attempt to pro- 
duce more reliable, valid and meaningful data. Of prime 
consideration would be the deve lopment of a more thorough 
training progrim for evaluation data collectors and inter- 
viewers . 

Phase II study design is diagrammed on page 81. 



PLAN OF PROGRESS 

Initial steps for implementing^ Phase II would con- 
sist of obtaining conf irma t ion^-'^-ff^ participating hos- 
pitals and consultants to continue with consultation for 
another year, replacing hospitals not wishing to parti- 
cipate with new hospitals, and recruiting additional 
consultants to replace those who would not be able to 
continue with the project. 

Training programs for consultants and local eduda- 
tion coordinators would commence in May 1968 and would 
continue throughout the remainder of thefirst year in 
one day segments. Training would consist of both indivi- 
dual sessions for ^ach of the two groups and joint ses- 
sions where content and interest overlap. 

Experiment al hospitals would be randomly selected 
into ''intensive" (five visits) and "minimal" (two visits) 
in such a way that each consultant would have a distri- 
bution of "intensive" and "minimal " hospitals. Con- 
sul tati on visits would resume in May 1968 and would 
continue through October for "minimal" hospitals and 
through March 1969 for ^."intensive" hospitals. Consulta- 
tion would reinforce and maintain worthwhile established 
programs , continue to stimulate program deve lopment based 
on assessment of hospitals* needs and interests, and con- 
tinue to develop the expertise of local physician education 
coordinators as hospital medical educators. September 15 
would be the official date .for transmittal of recommendations 
to hospitals, followed jby an eight month recommendation 
implementation period .V^During May and June' 1969, hospitals 
would be followed up by'^an indepth interview to determine 
status of both Phase I and Phase II recommendations. Data 
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on the nature of hospital changes occurring during 
Phase II would be obtained at the same interview. 
Other data pertinent to hospital and medical staff 
characteristics as well as frequency and nature of 
physician education activities would be collected 
throughout Phase II. During the second year of Phase 
II (June 1969 to May 1970) data would be analyzed and 
a report covering the activities of the three year 
contract would be written. (An overview of Phase II 's 
plan of progress is charted on page 83). 

PROJECT IMPJ.HMENTION 

RECRUITMENT AND SELECTION OF ilOSPITALS 

During May and June 1968 consultants visited Phase 
I exj '^rimental hospitals to explain the Phnsc II con- 
sultation progra:.') and to seek the ^.ospitals* commitment 
to participate in it. Specifically, discussions during 
the visits focused on the following areas: 

1. explanation of the program and elicitation 

of hospital commitment to participate in Phase II. 

2. the need for i )ocal hospital physician educat ion 
coordinator (preferable appointed for 2 years) 

3. the need for hospital commitment to financial 
support of- its education programs. 

4. the need for the hospital's cooperation and 
part icipat ion in res ear ch component of the 
study that is concomitant with consultation. 

5. discussion of the data pertinent to education 
activities that the hospital would be expected 
to supply. 

6. plans for a local education coordinator training 
program and the need to send the local cooroinator 
or representative to the first session on June 19. 

Of the 40 hospitals visited, one hospital chose not 
to continue in the study. Three hospitals, which had 
participated as controls in Phase I, were selected into 
the experimental group. The discussion to replace one 
with three hospitals was based on two factors: 1) all 
three hospitals when controls had urged PMI to accept 
them to receive consultation and 2) it was felt by the 
PMI staff that there were other hospitals within the 
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experimental group who would drop out of the study before 
the consultation program terminated. Four hospitals were 
solicited to participate as controls - three to replace 
the controls which became experimental and one to replace 
the control which dropped out of the project. (A listing 
of Phase II experimental and control hospitals may be found 
in Appendix D, p. D-2. See map, p. 20.) 

CONSULTANT RECRUITMENT AND HOSPITAL ASSIGNMENT 

The increased number of consultation visits required 
by Phase II*s design and the anticipated inability of 
several consultants to continue with the project neces- 
sitated the recruitment \nd training of additional con- 
sultants . 



Recruitment and contracting of consultants was 
accomplished by holding a dinner meeting for all pro- 
spective consultants, including both the 13 who had 
participated during Phase I and 8 additional physicians 
who had expressed interest in participating as consultants. 
Several members of the PMI staff outlined the philosophy 
and objectives of PMI, design of the contract, and 
specifically the nature of the consultation process under 
study. Preliminary results of Phase I and plans for Phase 
II were eported to the consultants. The responsibilities 
of consultants vis-a-vis time commitment fcr training, 
number of required hospital visits, and"*the intensive 
nature of half of the consultation relationships as 
modified in Phase II were carefully outlined before con- 
sultants were asked to make personal commitments . Nine 
of the 13 phase I consultants and S of the 8 new prospective 
consultants agreed to participate in Phase II. (A .list 
of Phase II consultants and the ir academic titles are 
presented in Appendix E , jjx E2a-b. 

Assignment of consultants to hospitals was based 
on" three dimensions: 1) distribution of hospitals 
with varying bedsize, 2) distribution of both intensive 
and minimal consultation hospitals, 3) geographical 
proximity of hospitals to each other. Care was taken 
not to disrupt relationshijiS established between con- 
sultants and individual hospitals during Phase I. Con- 
sultants were assigned to their prior year hospitals 
where feasible. Consultants wore assigned geographically 
proximate hospitals to permit exploitation of potential 
regionalization of education programs. Regionalizing 
education programs was felt to be a possible means for 
improving education programs at individual hospitals, 
and also for pointing up new potentials for improving 
patient care . 



ORIENTATION S TRAINING OF CONSULTANTS, EDUCATION 



COORD I NATORS & UVALUATIOK INTERVIEWERS 



Based on statistical results of Phase I, the ob- 
served and stated need for greater development of con- 
sultants| expertise, plus the continued need to better 
standardize the experimental treatment, that is, that 
each hospital receive comparable consultations, it was 
concluded that more adequate orientation and preparation 
of consultants should be undertaken. This tenet evolved 
into a formal consultant training program for Phase II. 
Training was extended to include physician education 
coordinators from experimental hospitals. The objective 
of this extension was to make local coordinators Setter 
able to cope with problems of continuing medical education 
on a sustaining basis. 

The concept of standardization of huiTS:!! variables 
via formal training was further expanded during Phase II 
of the contract to include a more rigorous interviewer 
- training program. 

The focal point of the present section will be the 
discussion of training sessions for all three groups. 
Consultant and physician education coordinator training 
will be discussed as a unit followed by a discussion 
of interviewer training. 

Consultant and Physician Education Coordinat ioli Training 



Following principles of adult education theory, the 
formal training program initiated during Phase II was 
designed so as to include both the transmitter and 
receiver of learning, i.e., knowledge of principles 
and methodologies of educational program development. 
A three fold rationale for this dual training component 
existed: 



1 . Education Standpoint 



The reader should note that terms DME , local • coordinator 
hospital coordinator, physician education coordinator will 
be used interchangeably in reference to the individual 
representing the hospital who assumed responsibility for 
education program . 



the 



Where learning needs of each group coincided, 
the provision of joint sessions would be more 
efficient and laore effective in terms of the 
potential for sharing a perspective on community 
hospital progran development that each group would 
bring to training. 

2 . Research Standpoint 

In order for the consultation process to have a 
comparable impact on all hospitals via the 
education coordinators (\vithin whom is vested 
primary responsibility for coordination of the 
education program at the hospital) similar 
exposure to various aspects of program develop- 
ment \:as deemed fruitful in order to avoid 
possible hiis engendered by exposure to one 
consultant only. 

3 . S ervice Standpoint 

Joint training would provide opportunities for 
a greater continiuty in the consul tant -consultee 
relationship and would provide consultants with 

opportunities to further assist their hos- • 
pitals in developing a program . 

Six training sessions were held during P'lase II 
(May *68-May '69), three of which were joint meetings for 
both consu Itants and coordinators and three of which in- 
cluded only one of the groups (two for consultants only, 
one for local coordinators to which consultants were 
invited and in most instances attended.) 

Added to Phase II training scssi on s was an evalua- 
tion component. Participants were asked to critique the 
session by completing a written evaluation instrument. 
The purpose of the evaluation was two fold: 

1. to assess the extent to which objectives 
were net; and 

2. to obtain information on desired content, for- 
mats and logistics of subsequent sessions. 

Tn this way both consultants and DME's were 
able to participate in the planning and organ- 
ization of each session. 



Results of evaluation will be discussed as part of the dis- 
cussion of each training session. 

Copies of agendums, training materials and evaluation 
instruments for each session may be found in Appendix 
Phase II, pp. F-4a-9c. 

Training Sessions »1 - June 19, 1968 (Appendix F-4a-f) 

The morning session was devoted to consultants only, 
while the afternoon and evening incorporated both the 
consultant and the local education coordinator. Trainee 
participants inc luded 12 PMI medical education consultants, 
and 28 local coordinators representing 26 experimental 
hospitals • 

The segment of the training devoted to the con- 
sultant dealt with a review and discussion of the con- 
sultation process aid procedures for implementation. 
Included were the following discussions. 

1 . Goals of Consultation Process 

Overall goals of the entire consultation pro- 
cess were reviewed: stimulation of staff to assume 
responsibility as a resource for education program; 
development of community hospital physician education 
programs based on full utilization of local resources 

(including manpower and facilities) and supplemented by 
external inputs as needed; and use of the Community hos- 
pital as a major focal point for continuing medical 
education of physicians. 

2 . Manifestations of Goal Attainment 

Measurab le indicators of attainment of con- 
sultation goals \\'ere discussed: appointment of a local 
coordinator; provision for financial support of education 
program (administration, staff, other sources, e.g., 
drug companies, foundations) ; allocation of secretarial 
assi.stance and program publicity relations; full identi- 
fication of local strengths and weaknessess; maximum use 
of local staff as instructors; use of outside expertise 
to complement and supplement local resources; an ^ initia- 
tion of regularly scheduled education activities (e.g., 
grand rounds, medical audit, CPC, guest lectures, etc.) 
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3 t Consultation Procedures 

Consultants \vere briefed on the importance of 
establishing rapport \^ith the hospital and especially the 
local education coordinator with whom he would be \\'orIcing. 
As an outside agent the consultant must be accepted by 
the hospital staff before any chansie may be expected. By 
the same token, however, consultants were cautioned ^o 
establish a non dependency relationship. An impo^^nt aim 
of the consultation was for the ^ ospital to function in an 
independent fashion after termination of the consultation 
process. To attain this end, the very important element 
of the establishment and development of a DME was again 
stressed. Functions of the DME were to assist and colla- 
borate with the consultant in developing an education 
program during the consultation program and to continue 
efforts of the consultant in aiding the hospital to develop 
programs independently I'^ng after consultation was withdrawn. 

Further discussions of the consultation procedure 
included the collection and use of data, the recruitment 
of local participation in program development, and formu- 
lation and transmission of consultant recommendations. 

The afternoon session, in which both consultants 
and coordinators participated, included a review of ^PMI 
philosophy and consultation procedures as well as the 
specific objectives of the contract. An analysis of the 
functions of a local . coordinator i.e., data collection and 
analysis, program production and program assess.nient, were 
discussed. Presentations by four consultants ot their 
consultation experiences illustrated the four vital aspects 
of the consultation process eluded to earlier in this report 
i.e., establishment of rapport, data collection, recruitment 
of local participants, and transmiti;al of recommendations. 
The remainder of the session was devoted to group discussions 
and exchange of problems and experiences at individual host)itals. 

In addition to the traditional educational methods 
of curriculum implementation such as lecture, question 
and answex, case history, and chalk-talk, the training 
utilized techniques of behavioral ipsychology such as 
sensitivity training exercises, group problem solving, 
and force field analysis. 



Evaluation of Training Session JL 



Analysis of responses from both consultants and 
DME * s to the questionnaire instrument pertinent to the 
June 19 session yielded the following findings. A 
majority of the consultants identified the DME^s pre- 
sence as the most useful part of the training session. 
Responses were of the following nature: "afternoon 
session with DME's participating", "contracts with DME^s^', 
"chance to see degree of acceptance by DME", "finding 
out what the DME felt about PMI", etc. Ranking second 
in the most useful categoi^y were "the sharing of ideas 
and • though ts , and general contact with .other consultants." 
As to the least useful segment of the training session, 
consultants seened to be in agreement that the morning 
session was the "least useful" part of the day. Responses 
were levelled at various areas: "excessive introduction", 
"repetition", "too much discussion that was not pertinent", 
etc., all of which were directed at the morning session 
consisting of the consultants only. 

Suggestions a3 to how the s-'ssions might have been 
changed to be more effective were numerous , yet all of 
them seemed to deal with a time factor, e.g., "set fewer 
topics to cover", "have fewer topics with more discussion", 
"less introduction - more time on crucial problems", 
"divide the morning session into small groups". 

Regarding suggestions as to agenda for future training 
sessions, consultants voiced a need for the following areas: 

Methods of evaluation 

Resources available to consultants for hospital 
programs 

Educational Techniques 

Specific information on regional resources, 
e.g.* RMP, medical schools, etc. 

Of paramount importance, however, was the voicing by 
consultants of a need to conduct and moderate their own , 
consultant session. In response to this need, the September 
28 session, which will be discussed subsequently, was given 
over to the consultants . 



Regarding the CME 's evaluations of the first training 
session, responses yielded the following. Consensus as to 
the Riost useful part of the session was the learning and 
sharing of other community hospitals' problems. The least 
useful segment of the session was the introductory philosopTTy . 
Criticisms were of the nature that it was either too long 
or too repetitive. 

Very few DME ' s had suggestions as to how they would 
have changed the session if they were in charge of it. 
The few opinions voiced were directed at requests for more 
case studies and more group work . 

Suggestions by local coordinators for future sessions 
encompassed all aspects of program development. The more 
frequent suggestions included: techniques for stimulating 
medical staffs to attend and participate in CME programs, 
methods for determining needs and program goals, and 
sharing problems and ideas with other local coordinators. 

PMI fashioned subsequent training sessions in response 
to all of the above findings. Special attention was paid 
to felt naeds of the participants. The reader will note 
that in response to requests for evaluation, an entire 
training session, that of February 1, 1969, was devoted to 
evaluation of the education program. 

Training Session #2'September 28, 1968 (Appendix F-5a-b) 

Training session #2, for consultants only, was he?d 
on September 28, 1968. Eleven of the PMI medical education 
consultants mot with the Institute's staff and several 
resource consultants. As indicated, in response to* the 
consultants' voiced need for moderating their own session, 
most of the day's planning and discussions were turned over 
to a consultant moderator. Rather than a didactic type 
of lecture session, the meeting evolved into a discussion 
and clarification and exchanges of feelings between PMI 
staff and the consultants. 

The essence of the entire session was devoted to 
various aspects and problems of the consultant's role. 
The relationship between the consultant and PMI, and 
between the consultant and his hospital were explicitly 
reviewed. Consultants' ambiguities regarding PMI's 
expectations and hospital commitments were clarified. 
Consultants expressed a need for knowledge as to what 
Postgraduate Medical Institute can provide in the way of 
service to individual hospitals. Areas in which they 



were especially interested included; nursing education, 
library development, audio-visual educational devices etc 
A further focal point of the discussion was the DME and 
what both the consultants and PMI might provide in the 
way of guidance to him. 

In conclusion, consultants expressed a strong desire 
for another meeting within two or three months, at which 
they could learn more about various educational resource 
areas, especially audio-visual presentational materials. 
In this way they felt they would be better able to offer 
more effective educational program consultation. 

Evaluation of Training Session #2 

As to the perceived usefulness of the September 
training session by consultants, 78% rated it useful or 
very useful. A good majority of the consultants voted 
the most useful part to be its clarification properties: 
clarification of the role of the consultant, clarification 
of expectations (what PMI expected from the consultant), 
etc. Other reported attributes contributing to the "use- 
fulness- of the sessions included "free wheel session without 
structured or biased agenda", "less gene " ization than last 
meeting", "improved communication betwec !I and consultants 

In regard to the least useful part of the training 
session, consultants voiced concern for more details, more 
specifics. One consultant perceived that discussions of 
audio-visual equipment was the least useful part only 
because there was lack of sufficient detail, specific infor- 
mation on the what's, how's, where's, etc. 

When asked of the focal areas preferred for future 
sessions, consultants responded with the desire for a 
sharing of other consultants' experiences, evaluation of 
impact on patient care, the use of auxiliary resources, 
role of RMP and available training for DME's. 



Training Session #5 - November 16. 1968 (Appendix F-6a-e) 

The session consisted of three components; a morning 
session for consultants only, a joint afternoon session for 
both consultants and local coordinators and an evening 
session for local coordinators only. Nine consultants and 
20 education coordinators participated. 
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The norning segment dealt with the consultant's 
relationship to his hospitals in terms of the experiences 
he had gathered, the problems he had encountered, and 
the insights he might have acquired. 

The format of the morning session was an open ended 
discussion. Consultants were given the opportunity to ex- 
press their feelings about their role as consultants. 
Several consultants expre ssed concern about their effective- 
ness at the community hospital level. This concern mani- 
fested itself in several areas, such as, were the consultants 
fully accepted by local hospitals, were they communicating 
effectively with the hospitals, etc. Consultants supported 
each other in their answers and solutions to the problems 
they had presented. 

There was a significant response when the moderator of 
this session asked the questions, "Having been a PMI con- 
sultant for six months, what have you learned, what do you 
know now that would benefit your future consultations, how 
can a consultant increase his competence and effectiveness?" 
One consultant responded with a proposal for live-in arrange- 
ments to be made for consultants so that they could become 
even more closely involved with the hospital. It was con- 
cluded that this would be a more appropriate role for the 
DME, but that the DME and the colisultant should work closely 
with one another . 

The afternoon ssssion, which included both consultants 
and local coordinators in a joint training session, consisted 
of brief formal lecture inputs followed by problem-centered 
workshops. The session was based on needs of the trainees 
as expressed by the trainees themselves in previous training 
sessions. In response to one of these expressed need''* for 
knowledge regarding resources, facilities and services 
available to the physician, PMI presented a range of topics, 
including : 

1. PMI-NERMLS* Core Library Development Program, 
and services avai Ip.ble . 

2. The Use of Audio-Visual Materials in Continuing 
Education Programs . 

Trainees were shown a f i Im' about appropriate and 
inappropriate uses of audio-visual techniques . 



♦Postgraduate Medical Institute-New England Regional 
Medical Library Services . 



3, Principles of Adult Education for the Physician 
Learner. 

A brief overview of educational principles 
in adult education was presented, (see Appendix F-6 
d for materials distributed), followed by presen- 
tations of a few case studies dealing with success- 
ful ongoing hospital-based continuing education 
programs which incorporated adult education tech- 
niques into planning and implementation. 

Following this brief conceptual input, the group 
was divided into three smaller workshop groups, each with 
a "resource person" as moderator, and each group meeting as 
a unit in separate places. The purpose of the workshops 
was to work through the application to the community hos- 
pital of educational resources and adult education prin- 
ciples discussed in prior presentations. 

After the allotted interval of 45 minutes, workshops 
were regrouped in order to provide trainees with further 
contacts with additional consultants and DME's. A "resource 
person" was available within each group to offer direction 
and guidance when requested and to clarify questions and 
ambiguities when needed. 

Upon completion of the workshop experience, the group« 
reconvened as an entire unit in order to review and share 
learning experiences. 

Finally, a wind down of the afternoon session con- 
sisted of an evaluation of the day, and a discussion of items 
for future training sessions. A consensus of opinion was 
reached to include the "evaluation of education programs" 
as a topic for a future training session. 

The present training session also included an evening 
session, which was devoted exclusively to the local coordin- 
ator, and which paralleled the consultant's morning session. 
Discussions, which concentrated on the role of the local 
coordinator, invited trainees to share with each other th^ir 
significant experiences, problems, and insights to date. 

Evaluation of Training Session '3 



As with all preceding training sessions, participants 
asked to evaluate the day's experiences along various 



process and content dimensions. In the pas^ the research 
staff had used such evaluations primarily as sources of 
learnings which might guide the planning of future training. 
To this ongoing process a second dimension was added which, 
it was anticipated would provide some measure of the Impact 
of the training on actual behavior change^ that is, scr»e 
indication of the training program's effectiveness beyond 
the anecdotal and test imonial evidence already accumulated . 
To this end, each trainee, consultant and coo iinator, was 
requested to specify one instance of a new learning arising 
out of the day's training which he thought he would attempt 
to apply in his capacity as consultant or local coordinator. 
Plans called for recording and cataloging the statements and 
for a review of the status of the indicated learnings with 
the author of the statements several months Irter to determine 
the actual amount of transfer and application to work situ* 
ations of learnings claimed at the time of training. ♦ It 
was hoped that this procedure would provide evidence to en* 
hance the persuasiveness of uhe already accumulated anecdotal 
and testimonial evidence supporting the effectiveness of the 
training program. 

Evaluation by the consultants of the morning sessions 
indicated that the ^^unstructured nature oi the conduct of the 
conference** was most useful as a learning format. Consultants 
felt that the most useful parts of the sessions were the dis- 
cussions on potential constructions of CME programs and on the 
possibility of a local coordinator accompanying a consultant 
on a visit to another community hospital. The liost frequent 
response to the question of desired areas for focus in future 
training sessions was information regarding resources avail- 
able to community hospitals. Other responses included "mech- 
anisms for maintaining the programs at the hospitals** and 
**e^aluation of the various approaches to continuing education. 
Common consensus as to the **least useful part of the morning 
session** was levelled at its repetitive qualities. Responses 
such as **rehashing of materials that had been gone over at 
previous meetings**, **reiterat ion during last hour**, ••excess 
talk**, etc., were elicited. Consultants had difficulty in 
expressing specific personal learning experiences that would 
be applicable in their role as consult^nts. Responses included 
••lack of awareness of fact that leadership can be programmed**, 
••only new suggest ion-possibil ity of another DME accompanying 



*Time and resource limitations of later phases of the contract 
precluded the follow up segment of this evaluation to determine 
whether reported learnings were ever actually applied. 
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consultant to hospital "profitable in bringing group 
together. (Can't single out specific points "no 
explicit learning event for directed use "question to 
be raised in future with DME at local level-what he 
would consider to be most useful type of consultant ", 
"need for personal study in educational principles and 
attempt at application ", and "increased sensitivity for 
limitations of capacity of community hospital to involve 
attending staff in self -education ." 

However, consultants were more specific in their 
statements of potentially applicable learning they 
acquired from the afternoon session. Responses were: 
"idea of exchanging review committees in evaluating and 
auditing, i.e., URC,", "discussion of facilities and 
services of Countway Library enlightening.", "sitting 
beside my DME counterpart and noting which points he 
thought of value. (Points he took notes on)", " tenta- 
tive set of principles provided by Ezra ", and "to insist 
on patient presentation at hospitals." 

Learning experiences of local coordinators were 
stated in more general terms. The nost frequently ex- 
pressed learnings pertained to the use of multiple 
physicians-representing various disciplines in presenta- 
tion of cases, the concept of the Core library and 
resources available through Countway Library of Medicine. 

Results of the discussion held at the end of the 
afternoon session and of the analysis of responses from 
the questionnaire as to desired future training topics, 
indicated that a subsequent training session should be 
concerned with evaluation of education programs. 

Training Session #4 - February 1, 1969 (Appendix F-7a-b) 

On February 1, 1969, a day-long training session 
for both medical consultants and local hospital coordinators 
was held at a conveniently located motel in the outskirts 
of Boston. The training session was a fourth in a series 
for consultants and a third in a series for local coor- 
dinators. Eleven consultants, 23 local coordinators, 5 
guest participants and 10 members of the Postgraduate 
Medical Institute Staff attended. 

In response to consultant requests at previous 
training sessions for specific content items in future 
sessions, the day's program encompassed various dimensions 



of the evaluation of medical education. No attempt was 
made to present a tight, pre-packaged, do^it ryourself 
evaluation technique. Rather, a variety of different 
approaches were demonstrated and hand-out materials 
pointed to the still broader realm of possibilities 
open to a prospective evaluator of medical education. 



Faculty for the session included: Thomas Durant, 
M.D., Massachusetts General Hospital; Clement Brown, M.D. 
and Daniel Fleisher, M.D., Chestnut Hill Hospital, Phil- 
adelphia; and r-ra V. Saul, Ph.D., PMI Consulting Director 
of Research. 

Content for the session covered principles of 
education evaluation, medical audit techniques, program 
design through need establishment, resources for evalu- 
ation, and local evaluation issues. Faculty members 
utilized various educational methodologies including : 
small-group process, lantern slide-assisted lecture, and 
round table discussions. 

Evaluation of Training Session ^4 

The February training session, like all others to 
date, ended on a note of its own evaluation. The con- 
sultants were somewhat polarized as to their reactions 
to the usefulness of the training session. On a 5-point 
rating scale indicating degrees of usefulness of the session 
4 of the 11 consultants rated it 1 or 2 (least useful) and 
6 rated it 4 or 5 (most useful) . 

Greatest criticism was directed, not so much at the 
content of the program, but at the format by which the 
content was presented. The consultants resisted a tech- 
nique to elicit active participation, a method they called 
•'playing games." They saw this as a ''waste of time" and 
"far below a physician's level of sophistication, interest 
and competency . " 

Local coordinators, on the other hand, had a more 
positive response to the training session. Only one of 
the 19 local coordinator attendees rated the session 1 
or 2 (least useful) and 13 rated the session 4 or S (most 
useful) . Of interesting note was the local coordinators 
responses to the most useful and least useful segments 
of the program. Dr. Brown's presentation on the building 



of an education based on identified need received 
both high and low ratings. The information, especially 
methods of data collection and staff involvement in 
setting standards and evaluation, proved to be more 
useful. However, the initial format of presentation 
was considered to be "wasted tiiiie," "too much semantics," 
"tedious buildup" "games." 

As stated in a previous report , such evaluations 
are conducted to the end of improving future sessions. 
One such improvement derived from the February 1 session 
evaluation was a decision to directly involve trainees 
in the design of future training programs. This decision 
was implemented through the formation of a planning 
committee composed of four medical consultants, two 
local coordinators, PMI's assistant director of research 
and PMI»s field director. 

Training Session #5 - April 26, 1969 (Appendix F-8a-c) 

The planning committee met several times and a plan 
emerged to hold an April 26 session focused on resources 
for continuing medical education. Although the program 
was designed for local coordinators, consultants were 
invited and 9 of them attended. 

A majority of the morning segment of the program was 
devoted to explications of institutional resources ^ 
available for continuing medical education in the New 
England area. Speakers representing New England medical 
schools included: Thomas Dawber, M.D., Associate Professor 
of Medicine, Boston University Medical Center; Herbert 
Constantine, M.D., Associate Professor of Medical Sciences, 
Brown University Division of Medical Sciences; Dean Seibert, 
M.D. Assistant Dean for Regional Affairs, Dartmouth College; 
and Daniel Federman, M.D., Assistant Dean for Continuing 
Education, Harvard Medical 'School. 

Regional Medical Program representatives included: 
Richard Chambex*lin, M.D., Maine Regional Medical Program; 
and Leona Baumgartner, M.D., Executive Director, Tri-State 
Regional Medical Programs. 

The morning session was rounded out by Kenneth Tcich, 
M.D. of the Professional Activities Study who discussed 
"Uses of PAS-MAP to Document Educational Needs." After 
lunch, in a session moderated by Robert P. McCombs, M*D., 
President of PMI, John Bjorn, M.D., demonstrated and dis- 



cussed "The Problem Oriented Medical Record as a Basis 
for Physician Education." The day's program ended with 
brief presentations and discussions of "real-life" hos- 
pital educational activities and techniques by several 
hospital coordinators . 

Mr. Norman Tucker of the Division of Physician 
Manpower , N . I . H . , a tt ended the day's program. His 
involvement included not only participating in the 
education sessions but also meeting with PMI consultants, 
local coordinators and staff for helpful discussions 
of proble-is relevant to the study. 

Applying insight gained from past evaluations re- 
garding the effective attention span of training sessions 
participants, the meeting was adjourned at 4;00 p.m. 

Evaluation of Training Session #5 

Reactions to the April 26 training session were, 
for the r.ost part, favorable. On a rating scale of 1 
(least useful) to 5 (most useful), all 9 consultants 
and 12 of the 13 coordinators rated the session 3-5. 
Of prime interest to the group were the presentations 
by the various medical schools describing thoir education 
programs and available resources, as well as the description 
of the application of the problem-oriented record. 

The presentation on the use of PAS-MAP to determine 
education need met with less enthusiasm. Of particular 
note however was the frequent response by participants 
that there was not one session that wasn't useful. Suggestions 
for future training topics varied. Included were: item 
and resident exchange with community hospitals; more locally 
successful method of audit and/or education; how to increase 
attendance and participation of the medical staff in post- 
graduate education at community hospitals; new ideas for 
formats of conferences -new subjects, new ways to present 
ideas, hov to organize, etc.; how to implement regional 
medical planning between hospitals in the same community.- 
esp. in areas of continuing medical education; panel of 
DME's who have university af f iliation-to sec how they view 
the role of the university in light of their own needs; 
staff s.el f -analysis -can a hospital staff function as a 
group?; and specifics of developing program objectives. 
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Training Session ^6 - June 10, 1969 (Appendix F-9a- c) 

On June 10, 1969, 12 consultants met for a dinner 
meeting to coordinate strategy for termination of consul- 
tation. Discussion centered on final hospital visits, 
c on ti nuance of the consultation programs, and recojiimend ac- 
tions for improving the consultation process. 

Consultants exhibited a polarity of opinions re- 
garding the competency and capability of physicians as 
education consultants , the preparation of consult ants 
and hospitals for consultation and the appropriateness 
of the current consultation mode 1 . Many of the consul- 
tants expressed feelings of discomfort in their role as 
educators, especially when the education process became 
more in depth at the hospitals such as during the second 
year of consultation. Some consultants felt that PMI's 
training program had helped the:n to be effective as edu- 
cators and that the program ought to be continued. A 
question was raised as to the efficiency of consultation 
in terms of the one to one interaction of consultant and 
his hospital especially in the context of a larger than 
40 hospitals setting. The b-ringing together of DME's 
and providing training for them^ could be just as effective, 
if not more so, than the provision of consultation. An- 
other consultant felt that meeting with just one or two 
members of the hospital staff on a visit was not enough. 
Built into the consultation should be an effort for the 
consultants to meet periodically with other staff members, 
as well as a periodic review of the consultation and 
program goals at a full staff meeting. Other suggestions 
were that possibly consultants should interchange hospita Is , 
that perhaps there could be a team of consultants working 
with one hospital, and that local coordinators or residents 
could be trained to provide consultation. Initially there 
should be field training under the supervision of the 
consultants. Another suggestion for improving the consuli- 
tation process was the development of a mechanism for • </ 
debriefing the consultant after his visit. This could be 
translated into somekind of a questionnaire which he could 
complete following his visit. Purpose of the form would 
be to enable the consultant to efficient ly document his * 
activities to help hira review and roctall all aspects of 
his visits, and to help him organize his next consultation 
st eps . 

Consultants varied in their opinions as to whether 



the hospitals had made a strong enough commitment to the 
"Consultation program. One opinion expressed was that the 
hospitals should be required to make some kind of financial 
comnitment. On the other hand, one consultant felt 
that the local coordinator, by taking time off to attend 
training, was exhibiting a strong commitment. Another 
consultant suggested that commitment might be for the 
hospital to expose its records for program need assessment. 
A question was raised as to what one does with a hospital 
who isn't willing to comply with commitments. It was felt 
that consultation should still be given, but that the hos- 
pital should be informed that it falls below the coinmitment 
criteria. Other consultants felt without a commitment to 
meet the requirements for receiving consultation, con- 
sultation would be ineffective and a waste of time . There 
was general agreement that hospitals should be properly 
prepar-d for consultation, that the entire staff should 
be r>ade aware of the program, that the local coordinator 
should lay the proper groundwork and that the board of 
trustees should consent to the program. 

Consu 1 tan t s received a suggested procedural guide 
for conducting their, final visit including suggested areas 
of discussion. (Appendix F-9b-c) 

Interviewer Selection and Training 

One outcone of Phase I was a finding of interviewer 
bias related to the mean number of reported hospital 
changes. (See Phase I^r^ge 69). Although this pro- 

blem did not invalidate the data, it was decided that 
furth er standardizati on of interviewer data collection 
techniques was necessary. An intensive training program 
was developed for the purpose of reducing future conta- 
mination . 

Criteria for interviewer selection were established 
and included: participation in the contract's first year 
interviewing; doctoral candidates in the behavioral sciences 
professional training in interviewing and observation, pre- 
ferably with so--ne experience in the field of hospital 
operations; and, the availability to participate in the 
complete research and training program. Seven interviewers 
were selected from two metropolitan Boston universities 
and one teaching hospital . 
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The initial training session for interviewers took 
place in conjunction with the April 26 training session 
for local coordinators.* Participants included : PMI 
research staff; several Phase I interviewers; and seven 
nev Phase II interviewers. The objectives of training 
were to equip the interviewer to effectively gather in- 
formation and relay hospital requests or questions to PMI. 

Training began with each interviewer receiving a 
sample interview packet and a presentation of PMI»s 
evolution and current activities. An historical and 
theoretical synopsis of the Public Health Service Project 
including an explanation of the research design followed. 
Using the design as the reference, the participants dis- 
cussed the practical considerations of specific inter- 
viewing situations and problems. Issues discussed 
included the handling of: 1) interviewee def ens iveness ; 
2) requests for group interviews; 3) hostility toward 
the Institute; and 4) interrupted or abbreviated 
interviews . 

The final consideration of the training session 
was a review of the materials in the sample packets. 
Included within these packets were interview arrange- 
ment and confirmation sheets, and interview guides, the 
standardized questions used for evaluating change and 
copies of recommendation forms where applicable. The 
guide and recommendation forms procedures were reviewed 
in detail so as to increase interviewer proficiency and 
reliability. Interviewers were requested to obtain 
missing information such as the Hospital Attributes 
sheets, and 1968-1969 Educational Activity Schedules 
or to make arrangements for their delivery to PMI.' It 
was suggested that hospital questions be written on the 
outside of the interview folders to facilitate immediate 
replies by the Institute. The program concluded with 
a review of administrative expense procedures. 

Part II of the interviewer training was observa- 
t ion-supervision of an actual interview. Each inter- 
viewer visited a hospital with an experienced interviewer 
from Phase I. The interviewer trainee observed the ex- 
perienced interviewer conducting two or three interviews. 
Rol^ were then reversed and the trainee administered an 
interview. Upon completion of the day's interviewing 
and data collection the trainee and his supervisor dis- 
cussed their experiences and analyzed and clarified 
interview procedure and technique. 



*oee Appendix F-lOa-'r for a copy of the training agenda 
and sam;ae interview packet. 
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The new interviewers then visited an assigned 
hospital on their oun. After all interviewers completed 
9 supervised and solo interviews * they met as a group with 

the research staff to discuss difficulties encountered 
and to suggest better methods of data collection. 

The staff found that the basic interview guide 
needed revision. Deleted from it was the section 
questioning the cause of specific changes. It was felt 
that the interviewers were far exceeding the time limit 
established for the interviews, and that the answers 
recorded for this section were subjective to the point 
of being unreliable. 

In order to reduce interviewer bias in hospital 
assignment , each interviewer was assigned to an approxi- 
mately equal number of experimental and control hospitals. 
Temporal and monetary economics of this task were con- 
sidered so that each interviewer usu?*lly visited more 
than one hospital in a geographic region. 

As a result of interviewer training and subse- 
quent research staff evaluation, one interviewer was 
dropped and his hospitals were redistributed. 



CONSULTATION IMPLEMENTATION * 

Phase II *s Scope of Work called for implementation 
of two levels of consultation: '^intcnsi ve'^ and "minimal''. 
Of the experimental hospitals, half were to receive 
"intensive consultation" (five site visits) and half 
"minimal consultation" (two site visits) , "minimal" 
being comparable to the amount of consultation al located 
for hospitals during Phase. I.. 

Consultants were scheduled to initially visit 
each of their hospitals during May-June 1968. Tliese initial 
Phase II visits were to be utilized to reestablish contracts 
and commitments to the second phase of the contract and, 
hence, to assure more effective functioning of all phases 
of activity. Specifically each consultant was requested 
to use the visit to cover at least the follov:ing points requisite 
to the onset of actual consultation visits later In the 
summer . 




*Case studies of two experimental hospitals are pre- 
sented in Appendix I, describing the hospital and the status 
of its education program at the onset of consultation, the develop- 
ment of consultation over the two year study period (Phase I and 
Phase II), the nature of the consultant's recommendations and the 
outcomes of consultation. 



It Obtain the name^ address, phone number, title 
Cpositlon) of local physician education co^ 
ordinatcr . 

2. Complete staff education questionnaires (See 
Appenc'ix G-8a) . 

3, Complete educational activities form (See 
Appendix G-2) . 

4, Orient local coordinator and hospital ad- 
ministrator to our expectations for the coming 
year in at least the following areas: 

a. Coordinator training 

b. Coordinator and adminstrator 1 

c. Local resources identification 

5. Ascertain what time schedule for consultation 
visits will make maximal impact on educational 
program development considering: 

a. When hospital prepares its annual 
budget so that support can be included. 

b. When key staff members will be avail- 
able for program planning. 

c. Optional strrting date for program. 

Inclusion of items four and five was specified 
so as to avert repetition of two Phase I problem areas. 
Since the staff was cognizant of the problems that may 
arise when the fall extent of participation requirements 
for training, administrative duties, etc. are not spelled 
out, underscored and reiterated, consultants were asked 
to be sure that all local coordinators and administrators 
were fully aware of all implications of their commitments 
under item four above. In the case of item five the staff 
was seeking to avoid, where possible, hospital visits and 
subsequent generation and transmission of recommendations 
at a time which might unduly prejudice chances of their 
successful implementation during the test period. 



Purposes pf subsequent visits Included stimulation 
of new program development, reinforcement and maintenance 
of established programs^ and encouragement and development 
of hospital sophistication in program evaluation* 

Consultants were requested to make final visits 
to their hospitals during the summer of 1969, prior to 
the onset of the fall education program. Consultants 
were to use such final visits to evaluate the quality of 
the present educational activities and to assist the hos* 
pital*s development of an education program for the coming 
year* Hospitals were to be encouraged to state their 
1969-1970 educational goals, objectives, and plans. In 
order to make the final visit more meaningful, each con* 
sultanc was equipped with reports of his hospital's im- 
plemented recommendations, changes, and other available 
information* (See Appendix pp« F-9b-c, Procedure for 
Final Visit.) 

Upon conclusion of his visits, each consultant 
was to prepare final reports for transmission to his 
hospitals summarizing the hospital's progress during the 
course of consultation and noting any final recommendations 
for program improvement. 

Fifteen consultants made 72 visits to 40 experi- 
mental hospitals under Phase II of the contract. (The 
co?(iparative figures for Phase I reveal that 13 consul* 
tants made 65 visits. to 40 experimental hospitals.) 
Twenty "intensive" hospitals received an average of 1.8 
visits, the range being 0 to 4. Twenty-two "minimal'* 
hospitals received an average of 1.6 visits, the range 
being 0 to 4. Table 13 illustrates the comparison of 
Phase I and Phase II hospital consultation visits. It 
must be noted that some consultants exhibited resistance 
to making final visits, claiming overcommitment of time. 
Other consultants felt that final visits were not warranted 
because of lack of hospital contact over an extended 
period of time disrupted continuity. Fifteen of 41 hos* 
pitals received final visits. 

A prime 'objective of consultation activities 
was the generation of recommendations to hospitals per<- 
tinent to the development of continuing education pro- 
grams. To that end consultants were supplied with a Guide 
Sheet for Preparation of Hospital Recommendations. (See Ap- 
pendix F-2b.) The guide was designed to provide consultants 
with a framework of basic considerations and potev^tial 
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^ Table 13 



COMPARISON OF PHASE I AND PHASE II HOSPITAL CONSULTATION VISITS 



I 
I 



r 



• 


PHASE I 






PHASE II 


• 








4. 


INTENSIVE 


MIMTMAI 


iXPERIMENJAL HOSPITALS 

• 


40 




42 


« 

20 


22 


EXPERIMENTAL HOSPITALS 
RECEIVING CONSULTATION 


40 




40 


19 


21 


CONSULTATION VISITS 


6S 




72 


36 


36 


VISITS/HOSPITAL: 
MEAN 


1.625 




1.715 


1.800 


1.636 


VISITS/HOSPITAL: 
RANGE 


1-3 




0-4 


0-4 


0-4 


CONSULTANTS UTILIZED 


13 




15 


15 


14 


EXPERIMENTAL 
HOSP ITALS /CONSU LT^ST : 
MEAJ^ 


3.077 




2.80 


1.333 . 


1.571 


EXPERIMEVTAL 
HOSPITALS/CONSULTANT: 
RANGE 


1-4 




2-3 


1-2 


0-2 



ERIC 



classes of rccQnjBendatipns 9 while restricting neither the 
nature nor the extent of consultants' recommendations. 
Thirtx«Nfiye of the 42 hospitals received a total of 295 
writen recommendations and 86 oral recommendations.* 
Of the 6 hospitals receiving neither written nor oral 
recommendations, 2 did not have consultation visits. 
One hospital withdrew from the study after the intial 
consultant visit , but before recommendations were generated. 
Table 14 comprises a comparison of the frequency of Phase 
I and Phase II recommendations. 

Essential problems, and their antecedent con* 
ditions, continued during Phase II. Some consultants 
were still not able to commit sufficient time at desired 
intervals to meet specifications of the study. As in 
Phase I, the results were frequent instances of delay or 
postponement of meetings with members of the experimental 
hospitals, restriction of the duration of consultations, 
approximately seventy-^f ive per cent participation in 
training programs, and finally tremendous variation in 
the time required by consultants to complete field reports* 
submit recommendations and then review them with the PMI 
staff. Only 8 of the hospitals received recommendations 
by the September deadline. 

In two instances consultants found themselves 
so overcommitted that they required assistance in ful* 
filling their assigned consultations. This, of course 
necessitated the recruitment and training of additional 
consultants, thus causing further delay. 

Of particular note with regard to Table 13 
is the row entitled **Visits per Hospital: Mean** which 
points to one consequence of the problem. Because con- 
sultants were not able to make the specified number of 
visits to hospitals, "intensive** hospitals received 
approximately the same degr*ee of consultation as the 
"minimal** hospitals^ that is, the designed difference 
between the two experimental groups disappeared during 
the actual implementation of consultation. It should be 
noted here that mid-way through the consultation period, 



♦Existence of oral recommendations were verified through 
perusal of consultant visit reports and personal **de- ' 
briefing** interviews with each consultant. 
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consultants werQ freed ojT "jiininar' and ^'intensive*' 
restrictions and tcM to nake as aany visits as 
feasible. This was done to speed up the generation 
of recomiicndations and to naxiaize the number of 
hospitals receiving visits* 



DATA COLLECTION AND RESEARCH INSTRUMP.NTS 

As discussed under Phase I of the present report » two 
categories data were collectec?: data for purposes of 
program development and data for purposes of evaluation 
of the impact of consultation. Phase II data collection 
embodied an entire review and reworking^ when necessary » 
of Phase I instruments and procedures. 



Data Collection fo. Purposes of Program 
DevTlopment 

Instruments utilized for program development 
purposes remained the same during Phase II. (See pp. 28 
-31 for descriptions of the instruments.) As a result 
of the first year's experience* it was decided that 
these instruments would not serve the dual function of 
obtaining information which could be used for program 
development and evaluation. Although consultants were 
encouraged to continue to colloct data via these in- 
struments to assist them in determining program needs 
and formulating recommendations /they were relieved of 
any responsibility for collecting information for 
evaluation purposes. 

Data Collection for Purposes of Evaluation 

Although types of duta collected during Phase II 
were essentially tho sams as that of Phase I* methods 
and instruments for collocting data were modified when 
deemed appropriate* 

Dependent Variables 



"Change" measures constituted the isajor dependent 
variable and were defined as follows: 



Hospital initiated changeshaving occurred 
within both experimental and control hospitals 
during and after completion of consultation 
program. ^ 

The instrument used to collect data was the 
Interview Guide Sheet utilized for Phase I 
data collection, but in a revised form for 
purposes of Phase II. (Appendix H-6-9) • 

The new form permitted classification of changes 
along a four point implementation Scale ranging 
from "fully implemented" to "just talk," Thus, 
analysis of change at experimental versus con- 
trol hospitals would be more precise in Phase 
II than in Phase I, Alterations -aiso occurred 
in the categories of inquiry included on the 
schedule. Changes included new categories, 
restructuring of existing categories , and 
omission of unproductive ones: 

a. Hospital budget category now reflected 
changes in categories of expenditure 
as well as dollar amounts. 

b. Patient care category now omitted in 
favor of more specific breakdowns in 
categories such as hospital facilities 
and services , equipment , and personnel . 
In the latter two cases the guide per- 
mitted differentiation of expansion 
and routine replacements. 

c. Medical staff attitude change section 
now focused on actual changes in be- 
havior in an attempt to obtain more 
rigorous data than in Phase I. 

d. Community change section was limited 
to hospital initiated changes for 
reasons similar to number c. above. 

e. Consulted hospital interview schedule 
now contained an added item to elicit 
the respondent's salient impressions 
of PMI's impact on his hospital. 



2*' Degree and rate of implementation hy ejcperi- 
mental hospitals of the recommendations 
transmitted by consultants. 

The reader will recall that a separate set of 
recommendations were transmitted by the con- 
sultant to the hospital during Phase II of the 
proj ect • 

Interviewers followed the same format as that 
of Phase I, i.e., a recommendation review form 
(discussed on page 38) was utilized to deter- 
mine degree of implementation of recommendations . 

However, in addition to determining rate of im- 
plementation for Phase II, recommendations data 
was also gathered pertinent to the present status 
of Phase I recommendations. This was undertaken 
in an attempt to learn more about the state of 
these initially transmitted recommendations after 
18 months (long term effect). Data enabled the 
research staff to examine the kind and frequency 
of recommendations maintained over an extended 
period of time, those continually rejected, those 
on which "delayed"actions were taken, and those 
which, although implemented initially, under«^ent 
attrition . 

As in Phase I, in-person interviews were employed for 
collection of hospital change and recommendation data. 
Selection and training of interviewers, however, was 
further standardized and improved in order to obtain more 
accurate, reliable and valid findings. (See Interviewer 
Selection § Training, pp. 100-102.) 

Sets of in-person interviews were scheduled and 
conducted between May 15 and June 20, 1969, at each of 
the eighty consulted and ncn-consulted hospitals to 
collect dependent variable data. Requests for the inter- 
views were made via personal letter to the hospital ad- 
ministrator approximately one month in advance of the 
desired date. PMI followed up the letter with a tele- 
phone call to confirm the interview appointment. Individual 
interviewers reconfirmed their appointments several days 
before their interviews. 
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Both consulted and non~consulted hospitals Cthough 
not contacted by P>1I during the preceeding months) 
provided excellent cooperation except in a few cases.* . 

At both consulted and non^consulted hospitals PMI 
requested individual interviews with the hospital ad- 
ministrator, the president of the medical staff the 
director of medical- education or his surrogate and the 
director of nursing. In most cases all four interviews 
were obtained in one visit though vacations and broken 
appointments necessitated some return trips. 

3. Comparison of frequency and scope of education 
activities during 1966-67 (pre consultation), 
1967-68 and 1968-69 (post consultation). 

In an attempt to obtain measures of educational 
activities pre and post consultation, of which 
efforts to collect pre measures at the onset 
of consultation via the consultants failed, an 
Education Activity Schedule was developed to be 
completed by the hospital, in particular the 
education coordinator, for each of the three 
years in question. (See Appendix H, p.H-11.) 



In most cases resistance to research efforts was limited 
to selected individuals within a hospital and seemed to 
be Idiosyncratic in causation. Thus, in one experi- 
mental hospital the DME refused to be interviewed, and 
in another inst-" tion both the president of the staff 
and the DME d--;- ed. Failure of the consultation to 
meet the DME's expectations seems to be the cause in 
the former case, while the latter's cause may be attri- 
buted to overall iii?dical s-taff resistance originally 
desired, and contracted for, by the administrator. In 
both instances of resis^snce cited above, as well as 
others less dramatically expressed, geography and local 
politics" also seemed partly responsible. Comments by 
hospital members to our consultants and behavioral 
science staff indicated that tht institutions in question, 
all located in the same state, seem to be increasingly 
oriented toward the growing medical school located in 
that state (at the expense of out-oi-state involvement) 



Data from the yearly schedules were to undergo 
comparative analysis to provide evidence of 
changes in both frequency and scope of education 
activities. Limitations in the method were that 
the data obtained for the first two years was of 
a recall nature, thus lessening validity, and 
second, the person who was in charge of the edu- 
cation programs for each year was not necessarily 
the one completing the schedule. Consequently 
the responder was dependent on records of pro- 
grams which may or may not be thorough and com- 
plete. To aid him in collecting the information, 
PMI research staff filled in available information 
pertinent to the hospital's education activities 
which had been provided by the consultants during 
Phase I. Responders were then asked to confirm 
the accuracy of the information and provide missing 
information. Upon review of completed forms, it 
was observed that the quantity of data reported 
was proportional to the amount provided by the 
research staff, that is, since PMI had more data 
on 1966-67 education activities than on the 
following two years, there was an unproportionate ly 
greater frequency of reported 1966-67 activities 
confirmed by the local coordinator . Another factor 
influencing the unevenness of quantity of .reported 
data was the willingness and "stamina" of the local 
coordinator for filling out all three forms (one 
for each year) . Although the three forms were not 
given to him at one time but were distributed to 
him in chronological order for completion at 
approximately 2 month intervals throughout the 
year, the task was time consuming and, was felt, 
his enthusiasm waned by the time form three (1968- 1969) 
came to him for verification and completion. 

Although. the data was partially analyzed, it 
was felt that any ^results yielded would be due 
more to the circumstance by which the task was 
accomplished and not to any real change occurring 
at the hospitals. Consequently, any further pre- 
senation of this aspect of the evaluation is not 
warranted . 

Independent Var iab les _ ^ 

Phase II independent variables also fol lowed essential ly 
the same format in content as those of Phase I . It 
will be recalled that independent var iab les were of 
a dual nature : 



1. Manipulated (or treatment) variable-that 
of consultation 



2. " Tiaed - (or classification) variables including 
hospital characteristics, medical staff charac- 
teristics, consultant characteristics, inter- 
viewer-interviewee characteristics . 

Alterations and modifications of consultation were 
discussed in a preceding section (see pp. 102-108.) 
Regarding "fixed" variables. Phase II of the project saw 
deletion of the interviewer characteristics and several 
additions to the consultant characteristics. Further, the 
wording and arrangement of variables were altered in 
several instances in order to make items clear, concise 
and meaningful, and subsequently maximize their probability 
of being answered in a valid and reliable fashion by the 
respondent . 

It will be recalled that the Hospital Attribute Form 
was the primary research form utilized for the recording 
of independent variables during Phase I. likewise, during 
Phase II the Hospital Attribute Form was utilized, however, 
only after having undergone revision. (See Appendix H, 
pp . H-5a-d) . 

Although format of data collection for independent 
variables remained essentially comparable from Phase I 
to Phase II, method of data collection was completely 
revised for Phase II. The reader will recall that for 
Phase I, it was the task of both the research staff to 
obtain the data from documented sources , and of the 
interviewers to verify or correct this information. with 
the hospital personnel at the interview scene. 

On the contrary, during Phase II a completely different 
procedure was followed. All information which was avail- 
able to the research staff regarding the various charac- 
teristics was put onto the newly revised Hospital Attribute 
Form. "Hospital Characteristics" section of the form was 
then mailed to the administrator of the hospital, while 
the "Medical Staff Characteristics" section was mailed to 
the director of medical education or, in the absence of 
such a person, to the president of the medical staff, well 
before the scheduled date of the in-person interview. in 
the control hospitals, the entire form was sent to the 
administrator. A set of directions asking the individual (s) 
to verify and/or correct the information provided and to 



supply missing inforjnation was included in the 
nailing, as was a cover letter explaining that the 
form would be picked up by the interviewer. It was 
expected that this procedure would yield more complete 
and usable data than was obtained during Phase I. 
Additionally, the revised method of data collection 
would ease part of the time pressure problem reported 
by evaluation interviewers during Phase I. 



PROCESSING OF DATA 

The essence of data processing (i.e., preparing the 
data for analysis by transformation of raw data to quan- 
tifiable data) was discussed in Phase I of the present 
report. Phase II data was processed or "transformed" 
along similar lines but with extensive modifications, 
revisions, and additions. Phase I data and results were 
merely tentative and preliminary, while Phase II efforts, 
on the other hand yielded more conclusive results. Phase 
II data, being more abundant and of better quality necess- 
itated more extensive data transformation. 

As In Phase I, the bulk of the present section will 
focus on a discussion of transformation of dependent 
measures, since a large portion of independent measures 
were either already reported in transformed form, trans- 
formation in many instances was synonymous with descrip- 
tive analysis, or many independent measures were merely 
••categorized'* to accommodate inferential analysis. 

Our discussion will begin with transformation of the two 
types of dependent measures: 1) data collected in both 
experimental and control hospitals regarding hospital 
initiated changes procured from responses to the Inter - 
view Guide Sheet and 2) data collected in experimental 
hospitals only regarding implementation status of Phase 
I and Phase II recommendations procured from responses 
to Recommendation Review Form. 



Hospital Change Measures Gathered Both 
in Experimental and Control iiospitals 



The first task vhich beset the staff was the 
question of validity , that is, were all reported changes 



tangible and quantifiable? Definition and criteria for 
valid reported changes were established . Valid hospital 
changes were based on aggregated reports of the admin- 
istrator, the director of medical education , the pres ident 
of the medical staff, and the director of nursing and 
were defined as any reported alterations or additions to 
existing hospital organization or operational function, 
including services, personnel, and facilities occurring 
during the period of May 15, 1968 to May 15, 1969. 
Changes which complied with one of the six following 
characterizations were classed invalid and excluded: 

1. Vague statements of behavior alteration which 
were not objectively measurable, e.g. , "working 
more closely", "gained experience", "more 
interested" . 

2. Unintelligible statements which could not be 
understood without supposition. 

3. Duplicate statements made by more than one inter- 
viewee (i.e., the ^duplicate was excluded) . 

4. Statements describing hospital responses to ^ 
extra hospital initiated activities, e.g., 
"an RMP representative visited the hospital". 

5 . Statements describing the continuation of 
activities begun prior to May 15, 1968. 

6 . Statements describing perennial activities , 
e.g., "hospital held its annual infectious 
disease symposium" . 

Next, criteria were established to permit valid 
counting of reported changes. A statement of change 
referent to a single item received a single score 
independent of indicated multiples of the item, e.g., 
"hospital ordered 3 electric beds" received one change 
score. If a change was stated as a generality with no 
specification of its components , it received one tally, 
e.g., "administration enlarged education budget". 
However, if a change was stated as a generality with its 
components individually specified, then the general 
citation received one tally and each specification re- 
ceived an additional tally, e.g., "administration in- 
creased education budget, that is, they increased 
physician education budget 15%, and increased nurse 
education budget 5%, received three scores; one for the 
general citation, and two for the additional specifications. 



To permit analysis of the implications of changes 
in the number of physicians on hospital staffs, all 
such changes were excepted from the criteria requiring 
that multiple items not receive multiple scores, e.g., 
••three internists were added to the staff," received 
three change scores , while "thre nurses were hired , " 
received one change score. When multiple individuals 
within a personnel catagory were participants in an 
activii:y only one change was tallied, e.g., "four inter- 
nists participated in an infectious disease training 
program," r^^ceived one change score. When multiple 
categories of personne 1 participated in an activity* 
then a change was tallied for each individual category 
e.g., "the administrator, DME, and four nurses attended 
the American Hospital Association meetings,'' received 
three change scores: administrator attendance received 
one change : DME attendance , one change ; four nurses 
attendance , one change . 

Reported changes were then grouped into the following 
categories indicating specific degrees of completion 
and behavior changes. 

1. Category A delineated changes which were ten- 
tative, futuristic or under discussion ("just 
talk"), e.g., "we are going to look for another 
internist", "we may utilize telephone consultation" 

2. Category B delineated changes in which the 
hospital exhibited definite commitment, or in 
some way indicated that planning had occurred, 
but in no way action had exceeded the level of 
commitment or planning , e.g., "we are seeking 
a new cardiologist", "the hospital has decided 
to buy a new X-ray machine". 

3. Category C delineated changes which were beyond 
the planning stage, but not yet concluded, e.g. , 
"a newly appointed director for in-service 
education is coming in the fall," "the hospital 
has ordered monitoring equipment". 

• 

4. Category D delineated changes which were com- 
pleted during the specified time period and 
were fully operational, e.g., "the hospital 
staff now reviews deaths weekly", "EEG*s are 
now done in the hospital". 



S. Behav i or Changes delineated joan i £es t signs of 
4iCtiyity by hospital staff which were modifi^ 
cation of the manner in which the behavioral 
pattern had previously existed. These were 
accepted as changes only if the reported ob- 
servations could be assessed objectively 
(quantitatively), e.g., "more staff are attending 
the educational program** was accepted as a 
valid change; "the doctors are more interested 
in films** was not accepted as a valid change. 

It will be recalled that categories of **educat ional** 
and **non-educational** constituted the major breakdown 
for types of change data during Phase I. Likewise* 
Phase II changes, were distributed into **educat ional** 
and **non-educational** categories, operationally defined 
the same as in Phase I. To be classified as educational, 
a change had to have a primary objective of affecting 
the education of hospital personnel, e.g., **implement 
grand rounds,** **initiate a team approach to education 
programs centered on the coronary care unit,** **medical 
staff and administration jointly appoint a DME.'* Any 
change that did not specify a primary educational ob- 
jective or was not directly related to the education of 
hospital personnel was placed in the non-education 
category, e.g. "develop an ICU,** **provide salaried 
emergency ward staff coverage,'* **invest igate the possi- 
bility of providing home care services." 

However, a further classification of kinds of 
changes was developed during Phase II. Content analysis 
divided hospital changes into six major areas pertinent 
to hospital function and structure - education, personnel, 
equipment, physical plant, selected specialty services 
and community activities. Assignment of a change to 
multiple categories was made when a change pertained to 
more than one category. Multiple scoring eliminate! the 
need to artificially limit a change's areas of pertinence. 
A detail breakdown of the major categories of change may 
be found in Appendix K. 

In preparation for quantitative analysis, data 
were numerically totaled indicating: total number of 
hospital changes, total number of educational changes 
(and percent of education of total) , and total number 
of non-educational changes (and percent of non-ed out 
of total). Data will be reported in the results section. 
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Consttltant Recommendations in Experiaontal Hospital^ 

Prior to analysis of written and oral recommendations 
validity criteria were established for their specification 
and classification • 

1. A valid recommendation must be independent and 
not contingent upon the implementation of other 
recommendations • 

2. A valid recommendation advocating encouragement 
and support of a proposed or pre-existing 
activity nust include specific methods or 
techniques . 

3. A valid recommendation pertinent to a pre- 
existing activity must suggest modifications 
of the activity. 

As with reported hospital changes recommendations 
were classified as educational or non^educational . 

Valid recommendations were characterized as educational 
if they net one of the following criteria: 

1. An educational recommendation must relate to 
individual or group educational activity for 
doctors, nurses, or paramedical personnel, 
e.g., "hold CPC once per month**, "discontinue 
monthly death conference", "send nurse to CCU 
training program at a university medical center", 
"use medical audit for educational purposes", 
••obtain more journals for library". 

2. Aneducational recommendation must relate to the 
appointment or inclusion of personnel to fill 
educational roles, e.g., director of medical 
education, director of in-service education, 
secretarial assistance for education, outside 
guest speakers . 

3. An educational recommendation must relate to the 
budgeting of fxj^ds for education, including 
allotment of salaries, equipment, space or 
facilities • 

4. An educational recommendation must relate to 
obtaining, retaining and/or training medical 



students, interns, externs or residents, e.g., 

"have staff physicians develop policies for use 

of privato patients in education of house officers. 

S. An educational recommendation must describe 

policies or methods to be employed in the form- 
ation and/or development of an educational pro- 
gram, e.g., "inform trustees of education 
program," "share responsibility for program 
among staff," "change by-laws," "plan to invite 
neighboring hospital staff." 

Valid recommendations were characterized as non- 
educational if they related to potentially non-educational 
hospital systems for which the recommendations did not 
specify an educational use, e.g., "develop an ICU," pro- 
vide computer management for accounting," "investigate 
the possibility of providing hospital home care service." 

In order to determine implementation levels of 
recommendations, the following criteria were applied. 
When the four hospital representatives interviewed 
unanimously agreed upon the status of a recommendation, 
it was appropriately classified as implemented 'I), 
rejected (R) or partially implemented (IPI) . When agree- 
ment concerning a recommendation was not unanimous the 
response of that- individual whose hospital function most 
closely corresponded to the nature of the recommendation 
was used according to the above classification. Data 
on the nature and frequency of Phase I and Phase II 
recommendations implemented will be presented in the 
results section. 



PRESENTATION AND DISCUSSION OF RESULTS 

Analysis of the dependent measures (or change data) 
gathered during Phase II were much more voluminous and 
extensive than such data gathered for Phase I. As a 
result, the present chapter will constitute a much mors 
extensive account of the impact of the consultation on 
educational program development. Discussion of the 
results of Phase II endeavors will begin with analysis 
of the following change measures: 



1. What was the nature ?.nd extent of Phase II 
recommendations transmitted by consultants 
and implemented by the Aospitals? 

2. What was status of Phase I recommendations 
after an 18 month implementation period? 

3. What was the nature and extent of Phase II 
changes occurring in both experimental and 
control hospitals? 

Discussion of a comparative analysis undertaken 
on hospital chanf^es in experimental and control hos- 
pitals will follow. 

A subsequent secJCion will discuss utilization of 
the above dependent measures in determining the effect 
of the various "fixed" independent variables (i.e., 
hospital characteristics , medical staff characteristics , 
and consultant characteristics) on the nature and degree 
of change. This latter section is derived from the 
ques^tion posed by the contract objective: What factors 
facilitate or inhibit implementation of education' 
consultation at community hospitals? 

ANALYSIS OF TRANSMISSION AND IMPLEMENTATION STATUS 
OF RECOMMENDATIONS 

Transmission of Recommendations 

A quantitative summary of recommendations trans- 
mitted to experimental hospitals receiving education 
consultation during the two phases of the pro j ect 
(1967-1968 and 1968-1969) appear in Table 15. The 
table differentiates between recommendations that were 
transmitted in written form (which incidently had a 
more systematic preparation and review by both the 
consultant and the Postgraxiuate Medical Institute) and 
oral recommendations transmitted more inf ormal ly during 
the course of conversations and visits between the 
medical consultant and the hospital staff. The most 
obvious inferences that can be made from the data are: 
1) oral recommendations occur much less frequently than 
written recommendations, and 2) the most frequent con- 
tent of recommendations pertains to educational in 
contrast to non-educational issues, i.e., 318 written 
recommendations in 1967^1968 and 278 written recommen- 
dations in 1968-1969 of a combined total over a two 
year period of 740 recommendations which are pertinent 
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r« » till! for»at of educational 
to educational aatters te.g-. 

programs 9 etcO^ 

Purther examination of Table IS^inaicat^^appr^ 

■ately the »*^\""»i*^ls irPhase II (X.9.83) as 

were transmitted to ^^^pitals in Kna ^^^^ 
were ".ns.itted in Phase I X 9.81^ ^^^^ 
appear to be significantly di received 
approximately 1/6 te^cr . contrast to 

.??t«n «""'-*n:°i;osilt.u"ch". ?o"rop out and/or 

l"rnS!;S.":r««:«nd«lo«%rr?:inin? to .duc.tlon.. 
" „r.duc«ioJ.l content .nd ,la th. for„.t, o£ 
"u?in versus oral communication were manifest. 

r.^-.T-i .nn of Degree of Implement-tlon — 
• of fiiase "T~anj Phase II Rec onnondations 

fr.auency aecomnendat i ons Undergolnt Complete 
ipplemer-atlOP 

enc.s^^^;;rurfn,Ifn^r.^?r:ruHTec'o:m^Sd:tl;^r 

'tiIt'o?- tSe'3ll"di«rioSal recommendations transmitted 

Phale l! a total of 118 (J7 ») ««" completely 
? , .t.^t the time of the ensuint field survey 
Wur"""ths after th. transmission date of the 

i„omme.datlons) . ■'"'"'"''"''idAVonV that were fmple- 
educationally oriented reconnendations that were i»pj= 
^.n«d bv a iiven hospital was 3. J and the number 
Tf "ospi als'ii"lved'ln this portion of the sample was 

Jial f equency of oral to written recommendations. 

Since data are relatively spa«e for non-educational 
and/or Sral reconnendation. caution should used 
SJaiing inferences that the apparent differences m rates 
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of complete implementation o£ the various categories 
of recommendations being transmitted are statistically 
significant. What does stand out as a provocative 
finding is that the rates of complete implementation 
for recommendations transmitted during Phase I is 
considerably lower than the rates of complete imple- 
mentation of recommendations transmitted during Phase 
II. There are alternative hypotheses to explain this 
apparent difference. Among them is the fact that the 
duration of the available implementation period for 
recommendations transmitted during Phase II was 
approximately 7 months in contrast to an implementation 
period for recommendations transmitted during Phase I 
of approximately 4 months. Al o, the differential rates 
of completed implementation may have been due to a 
greater degree of "readiness" in the second year on the 
part of the institutions receiving the recommendations 
in contrast to their "readiness" to implement recommen- 
dations during the first year of the study. A third 
possible explanation could relate to the drop-out of 
or lack of cooperation to collect data by six hospitals 
in the second year of the study. These hospitals hrd 
proportionately lower rates of implementation of first 
year recommendations. However, it would seem to be 
most reasonable to note the relative increase in 
complete implementation rate during the second year in 
contrast to the first year of the study. The data in 
this table should be considered with reference to 
Tables 1/and ISwhich follow. 

frequency of Partial Implementation of Reco mmendations 
Transmitted to Hospitals 

Table 17 contains a summary of recommendations which 
were reported by hospital personnel as being partially 
implemented between the tim.e of receipt of th e recomme n- 
dation and the time of the field interview. In the case 
ot Phase I recommendations the implementation interval 
was approximately four months, and in the case of Phase 
II recommendations the implementation interval was 
approximately 7 months. An illustrative reading of the 
table indicates that of the 318 Phase I recommendations 
pertaining to education, 64 C20%) were partially implemented- 
an average of 1.7 partial implementations of such P'^""^"^*^^' 
educational recommendations by the 36 hospitals involved. 
Again, the relative infrequency of non-educational 
recommendations (both written and oral) and their partial 
implementation suggests caution in their interpretation 
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Perhaps the most salient finding reflected in Table 17 
is the highly depressed rate of partial implementation 
of transmitted recommendations during the Pkase II period 
in contrast to the Phase K Again the same three al^ 
ternat ive explanat ions may account for the finding , i.e., 
hospitals may be more ready in the second year to com- 
pletely implement (less likely to do a partial job), 
hospitals likely to do a partial job have dropped out 
in the second year, and lastly, since more time is 
available in the sec on d year, f ewer jobs will be left 
incomplete » 

Combined Frequency of Fully and Partially Implemented 
Recommendations 

Examination of Table 18 provides a fuller opportunity 
to draw inferences relating to total implementation (a 
combination of partial and completely implemented recom- 
mendations) . The pattern here is somewhat clearer than 
in the prior two tables, in showing that the approximate 
implementation rate of Phase I educational recommendations 
was approximately 55% in contrast to an implementation 
rate of approximately 63% for Phase II. Somewhat similarly 
the combined implementation rate for written non-educational 
recommendations was approximately 49% for Phase I in contrast 
to approximately 94% for Phase II. (Caution should be 
used in the interpretation of this marked difference in 
light of the relatively small nur,ber of cases involved.) 
Also, the net impression of a higher frequency of imple- 
mentation of recommendations transmitted during Phase II 
is maintained but not as drama t ically as are the comparisons 
manifested in the examination of either complijtely imple- 
mented or partially implemented recommendations . The same 
hypotheses cited earlier may account for the apparent dis- 
crepancy. 

Implementation Status of Phase I Recommendation 
at the End of 4 and IS Months 

As discussed in a preceding section, during Phase II 
interviews, data Were gathered not only on the rate of 
recommendations implemented for Phase II recommendations 
but also pertinent to the status (18 months after trans- 
mittal) of recommendations transmitted during Phase I. 
The following discussions will center around the im- 
plemejitation status of Phase I recommendations at the 
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end of Phase II C18 months after transmittal) accompanied 
by a comparative inspection of the same recommendations 
at the end of Phase I (4 months after transmittal) • This 
dual inspection of the same recommendations was undertaken 
in an attempt to determine both short and long term effects 
of consultation. 

Examination of Table 19 permits us to set the stage 
for an aniilysis of changes in the status of implemented 
and rejected Phase I recommendations throughout the 
period of Phase II- Table 19 states in essence that of 
the 353 recommendations transmitted in Phase I, a total 
of 191 (56%) were partially or completely implemented, 
within four months after being received by the hospitals 
Furthermore, 151 recommendations were not implemented 
in any manner at the end of the four month implementation 
period . 

Inspection of Table 20 permit^ the reader to note 
the long term status of initially implemented or rejected 
recommendations- It will be noted that the summation 
of/ the recommendations fully and partially implemented 
anti' the recommendations rejected do not total to the 353 
transmitted. The 16 recommendations in the no response 
column relate to a small group of hospitals which did not 
choose to continue participation in the study during the 
second year and/pr which data pertinent to the 18 month 
status were not available. (More will be said elsewhere 
about such attrition and the significance thereof) • For 
the current purpose it indicates that 334 (95%) of an 
originally transmitted 353 recommendations involving 35 
hospitals represent the body of data on which change in 
implementation of recommendations can be focused. An 
illustrative reading of the- table might proceed as follows: 
of 126* initially and completely implemented recommendations. 



* 126 includes all initially completely implemented re- 
commendations of which there were implementation data 
at the end of 18 months (130 minus 4). 
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96 remain completely implemented after 18 months, whereas 
four are now partially operational and 26 have bteen 
terminated completely and rejected. Wher the data in 
Table 20 are compared with that of Table 19 , one can 
begin to infer that approximately 75% f96/126) of the 
initially (completely) implemented recommendations sur- 
vive for a period of as long as 18 months after their 
initial implementation in contrast to approximately 25% 
of such initially (completely) implemented recommendations 
which fail to survive. 

Similarly, one can say that of the intially partially 
implemented recommendations (N:=69) , five (7%) continue 
to be implemented in partial form, 40 (58%) have becomed 
implemented fully, and 24 (35%) have been dropped com- 
pletely by the end of 18 months. It is interesting to 
note that the recommendations which are only partially 
implemented initially have a greater likelihood of being 
implemented fully by the end of an 18 month period than 
of being terminated and dropped. 

Likewise, further examination of Table 20 indicates 
that of the originally rejected 139* recommendations, 74 
continue to be rejected (i.e., have not been implemented 
either partially or wholly). However 64 have been com- 
pletely and 1 has been partially implemented. The inter- 
esting clement here appears to be that there is about a 
46% chance for a recommendation which has been initially 
rejected to become fully implemented during a follow up 
period of approximately 18 months. 

Of particular interest in this Table is the fact that 
although the data derive generally from 35 hospitals who 
continued to participate in the study, it should be noted 
that the variation in numbers in the extreme left column 
indicates that somewhat less than the 35 hospitals are 
involved in that class of recommendations. For instance, 
the appearance of an N of 30 in the left column, pertaining 
to completely implemented I'ecommendat ions , moans that five 
out of the 35 hospitals did not implement completely one 
or more of the recommendations transmitted to them. In 
analogous fashion the appearance of N*30 in the cell per- 
taining to initially rejected recommendations indicates 



* 139 includes all initially rejected recommendations of 
which there were 18 month status data. (151 minus 12). 



I 

I 

I that there vere five ^hospital s vho received recominendations 

' and who failed to reject one or jnore of the received 

recommendations. In a sense this latter group of hospitals 
iay Se seen as a class of "open-minded", "eager to change", 
"change prone", etc.. hospitals, in contrast to the five 
hospitals vho failed to completely implement one or more 
of the transmitted recommendations and which may be charac- 
terized as "resistant", "reserved", "non-change prone', 
etc. . hospitals . 

All in all. the salient implications of Table 20 

are that the delayed status of i"^^ . ttkeH 

commendations (completely or partially) is m all likeli- 
hood for a continued and further implementation and that 
even in the case of initially rejected recommendations a 
substantial number do ultimately get implemented. 

Table 21 has been prepared as a derivative of Tabic 20 
in a manner to permit an assessment of the fate of recommenda- 
tions transmitted in Phase I after a history of eighteen 

months listed under such headings as con.tlJl!ifeA..-r. g . j . fi < ^ ^ ^ fiP > 
inrreafied im plementation , maintenanc e and attrition. ine 
estimate of continued rejection is simply the number of 
recommendations initially rejected (not implemented 
which at the end of an eighteen month period are still not 
implemented (even in partial fashion). Table 21 indicates 
that approximately 55% (74 of an initial 151 rejected 
recommencations) are still rejected at the end of an eighteen 
month period Another way of interpreting this number is to 
examine the number of continued rejections in contrast to 
the total number of recommendations transmitted U-e.. /4 
over 334^ which when computed indicates a 22% rejection rate 
of all transmitted recommendations or a 78% implementation 
rate (cor.plete plus partial). 

Similarly, examination of Table 21 entitled hlaint enance_ 
indicates that a total of 101 initially implemented (either 
comoletelv or partially) recommendations persist for the 
duration of the eighteen month period. This observation may 
also be characterized in terms of the percentage of recom- 
mendations continued in their implementation in comparison 
to the total number of recommendations initially implemented. 
In this particular instance this would be 101 over 195 
or approximately 52%. 

Exar.ination of Table 21 , on the other hand, indicates 
that in the case of 105 recommendations which were initially 
rejected or partially implemented, increased implementation 
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Table 21 



COMPARISON OF 4 AND 18 MONTH IMPLEMENTATION 

STATUS OF RECOM'lENnATIONS ACCORDING TO 
CONTINUED REJECTION, NLMNTENANCE, INCREASED 
IMPLEMENTATION AND ATTRITION 



• 

\ CONTINUED 
; REJECTION 


RECOMMENDATIONS 
TOTAL # S % 


TABLE VII A 

INITIALLY AND CONTIN 

(N = 30) 

MEAN #/hosp. 
CONTINUALLY REJECTED 


UALLY REJECTED 
S.D. 


4 

t 
t 

'} 

# HOSPITALS 
CONTINUING TO REJECT 


74 (53.24%) 

139 


2.47 


2.38 




i 
i 

1 

i 

r 

; MAINTENANCE 
' OF INITIAL 
i (FULL OR 
j PARTIAL) 
I STATUS 


RECOMMENDATIONS MAINTA 

RATIO OF NUINTAINING 
STATUS TO INITIAL 
IMPLEMENTATION 


TABLE VII B 

LINING THEIR INITIAL 

(N = 31) 

MEAN #/hosp. 
MAINTAINING 
STATUS 


IMPLEMENTATION STAT 
S.D. 


ti 
• 

us \ 

\ 

» HOSPITALS 
MAINTAINING INITIAL ;: 
IMPLEMENTATION STATUS '.• 


101 (51.79%) 
•195 


3.26 


2.21 


•i 

28 :1 

'«*• 
J 


; INCREASED 
! IMPLEMENTATION 
■ ON INITIAL 
STATUS 


RECO.\WENDATIONS INC 

RATIO OF INCREASED 
TO INITIAL 
IMP.. MENTATION 


table vii c 
:reasing initial impi 

(N = 33) 
MEAN #/hosp. 

increasing 
implementation 


.EMENTATION STATUS 
S.D. 


# HOSPITALS ? 
INCREASING i 
IMPLEMENTATION STATUS ' 


105 (50.48%) 
208 


3. 18 


2.67 


30 1 
« 


• 
• 

1 

1 

i ATTRITION 
, OF INITIAL 
\ STATUS 


REC0.\1MENDATI0NS REGRE 

RATIO OF REGRESSION 
TO INITIAL 
IMPLEMENTATION 


TABLE VII D 

:SSING FROM INITIAL 1 

(N = 31) 

MEAN #/hosp. 
REGRESSING 
IMPLEMENTATION 


MPI.EMENTATION STATU 
S.D. 


•4 
f 

s ; 

» HOSPITALS ■ 
REGRESSING FROM \ 
INITIAL i 
IMPLEMENTATION STATUS ' 


54 (?7.69%) 
195 


1.74 


1.57 


22 
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took place Ci*e,, changes occurred from initially rejected 
to partial or full implementation or from partial imple- 
mentation to full implementation). Again, if this rate 
of increased implementation is compared with the total 
number of initially rejected or partially impleiaented 
recommendations approximately 50% (105/208) of these 
rect>mmendat ions increased in their implementation or 
thc^r degree of implementation during the eighteen month 
period . 

Finally, examination of Table 21 entitled Attrition 
reveals that there were a total 54 recommendations which 
initially were fully implemented or partially implemented 
and which during the course of the eighteen month period 
shifted to a lesser degree of implementation (i.e., fully 
implemented to partially implemented, partially implemented 
to no implementation, fully implemented to no implementation). 
Appropriate computations with the data in this category 
indicate that approximately 28% of initially implemented 
recommendations underwent reduction or termination . 

Table 22 represents another way of organizing the 
data relating to the continuity of implementation of the 
recommendations. In this table the data have been organ- 
ized so as to ignore initial states of implementation and 
to examine the status o£ the 334 transmitted recommendations 
at the end of the eighteen month period of possible imple- 
mentation. The data here essentially indicate that 200 of 
the 334 recommendations or approximately 60% were imple- 
mented fully at the end of the eighteen month period. Ten 
recommendations of the 334 or approximately 3%^ of the total 
recommendations were partial ly implemented. Final ly, 
approximately 37% (124 of 334 initially transmitted recom- 
mendations) were not in a state of partial or complete 
implementation, hence, must be considered rejected. This 
latter set of data is perhaps the source of the best recom- 
mendations regarding potential expectations during the 
course of such educational consultation as has been described 
in the present report, i.e., all things considered, a full 
implementation rate of approximately 60% might be expected. 

As way of summary, Table 22 indicates that first year 
recommendations were followed up twice - four and 18 months 
after they were submitted. Second year recommendations 
were fol lowed up once - seven months after submission. 
Within four months 37% percent of first year recommendations 
were fully implemented. An additional 19 percent were partially 
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implemented, that is, action to implement recommendations 
had been initiated but not yet completed* Forty^three 
percent of the recommendations vere rejected • 

As discussed earlier, field evaluat:)rs reported that 
hospitals w^re often unable to implement recommendations 
within four months after submission. For example, pro- 
blems such as the time needed to obtain staff approval or 
to get trustees or administration to allocate funds, often 
prevented full implementation within the four months. ^t 
was concluded that a longer follow-up might produce more 
valid indications of the hospitals' ability to act on 
consultants ' recommendations . This notion is sup ported 
by findings summari zed in the 18 -month f ol low-vp column 
of Table 22 which shows that full implementation increased 
from 37 percent to 60 percent at the end of the IS months, 
that partial implementation dropped from 19 percent to 
3 percent, and that recommendations rejected decreased 
from 43 percent to 37 percent. Though there were movements 
in all directions among implementation status categories , 
the net effect was that after 18 months the partially 
implemented category disappeared while the fully implemented 
category increased markedly and the rejected category 
stayed essentially the same. The second year's seven-month 
implementation period yielded results comparab le to those 
found at the end of the 18-month follow-up of the first 
year 's recommendations . 



Content Analysis of Consultant Recommendations 

Phase I and Phase II recommendations were content 
analyzed to determine what kinds of recommendations wot*e 
more frequently implemented or rejected, what kinds 
recommendations were more likely to stay implemented over 
an extended period of time, as well as the inverse question 
of what kinds of recommendations, after an initial imple- 
mentation , are then re j ected . The content breakdown of 
Phase I recommendat ion«* after a 4 month implementation 
period was presented in earlier discussions Cpp« 53-56 )* 



Content Analysis of Phase I Recommendations After 18 Mo nths 



Table 23 presents a content analysis of general 



NttatricAl Suaaarx of the laf Icacnt at ion Statu* of General Cat«» rie» 
•f fhm%9 I KecotaenJations at the En4 af 4 and it Months 



^OTAL KUMICR OF «CO*! :rNO \T IONS 




EDUCAT 1 OHkL R CCOMMC'.P U 1 0.SS 



A. rHYSICIAS E?UCAT10% atCOVIENDAT IONS 
i, EDUCATIONAL r.'OG' ' RCSOURCCS 



Financial Support for rro^ran Dcvtlopatnt. inciuding 
Hospital. Staff, and Other Sources 



Educational Facilities snd Materials, including Audio- 
Visual Aids and Librar> Provisions 

Hospital Personnel, including DME. Pathologist. Secretaries 
and librarian 

Eatcrnal Educational Resourtes.^ ificludiii; Education Agencies. 
Visiting C'insultants . and Outssde Speakert ^ 



Regional Interhospital Collaboration, including DME. 
Funds. Lectures and Tr^chm; PersonntI 



EDUCATIO.NAL PROGRAM PLAX2«1NG 

a. Progran Assessacnt. includini£ Evaluation of Prograas. 
Staff Needs and Interests, and Coaaunity *>rcds 



k. Prograc Design, including Educational Coals. Structuring of 
Activities, and Rotating Rc stons tbi 1 i tv for Eitucational 
Planning and I^nlof^cntat ion j'ong S;aff 

C. Scheduling anj Attendance o: iducational Pro;raos.~ including 
Pre;>arAtion and Cahlicitv o: Tro^raas. and Keeping 
Attendance and Minute« vt' Vcetings 

3*^ CDUCATIONAI rc-M HXM PM-I I *ti MAI his ^ 

a. patient t>T Case Centered tducation Activitie«. including Cr^m^ 
Rounds. crc*s. Case Pre scntztt ions . and Case Literature kcvie** 



Sub^ect-oricntcd Fducation Activities. incStiJinc Cour>cs 
anj Lecture I'ro^^rict. liiscusMon Croups, and Specia Itjfcare 
tdttcation ITo^r^.as 



Routine lio^;>it;»I Meetings (kusmess anU Connittce) and 
the education rro;:r37* 

House 'Officer Training, including Resident and lntert> Trainir.:- 

Miscellaneous T.ducation Pro;:rt"s. including Self -assessRcnt 
Tests and Sending rhvsicians Out for Tosigraduate Training 



XURSISG AM» ALLIL;» hCALTU PrCO'f^LN.-AT 10%S 



RESOURCES, including director of In-service Teaching. Vistting 
Nurse Consultants, and Ir.ter-hospttal Director of Sursing 



2. JOINT PHYSICtAN-*:*j;:Sr education activities, including Mutual 
Phjrsician-Nurse Planning and Trograas 



ESTA»L1SIi*:LNT of irl'SritViCr TdvCATIOn;. mcluJin; Teaching of 
Nttrses bx Medical Staff anJ Tro^raas for Fa r aned i c a t Persnnntl 



II. NON'EDUCATION ilCC0*tML*>4;AT ; OSS . includinc Making l^ider Use of Ecergcncy 
Roon. Establishing Clinics. TsLiu I t »hing Full or Part:a:ix Paid 
Departnoat Chiefs. Developing Plans Relating to Regional Serrices 
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categories of Phase I recommendations after 4 and 18 
months of possible implementation. (A more detailed 
content breakdown may be found in Appendix T,pp. J-2a-fo 
The 4 month implementation period columns indicate^ 
the frequency of kinds of recommendations that were fully, 
partially or not implemented at the end of that time 
period. The 18 month columns categorize recommendations 
by their naintenance (continued), increased implementation 
or attrition status*. 

Of particular rote with regard tj the 18 months im- 
plementation status is that the continued partial implc - 
mentation category virtually disappears. Recommendat io/i's 
that were partially implemented at the end of 4 months have 
either been fully implemented or rejected at the end of 18 
months . Most common recommendat ions continuing to be fu lly 
implemented are those in the area of physician educational 
program planning (95%, 20/21), specifically those recommenda- 
tions pertaining to education program assessment and those 
dealing with mechanics of program implementation such as 
scheduling and ^publicity activities. Nursing and ?.llied 
health related reconmendat ions also have a high incidence of 
continued full implementation (100%, 9/9) at the end of 
18 months. External educational resources for program 
development such as the use of visiting consultants and 
outside speakers is yet another area of high continued 
full implenentation (86%, 12/1 t). The lowest frequencies 
of continued full ir:plementat ion appear to fall in the 
classifications of non-education recommendations (50%, 
6/12), financial support for program development (57%, 
4/7) ^educational facilities and materials (56%, 5/9), 
and impler.entation of sub j ect -orient ed education activities 
(43%, 3/7). Caution should be made in interpretation of 
some of the data as some results are based on low frequencies. 



♦Maintenance those recommendations maintaining the same 
implementation status at the 4 and 18 month assessment 
periods . Increased iniplementat ion - those recommendations 
either partially inp^^mented or rejected at the end of 
4'jnonths that becane fully or partially implemented at 
the end of 18 months. 

Attrition - those recommendations initially fully or 
partially implemented that became martially implemented 
or rejected at the end of 18 months. 
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Recommendations Khich tend to remain rejected after 
18 months are highest in the classifications of regional 
interhospital collaboration (ys'i, 7/9), program assessment 
(75^5, 3/4), resources for nursing and allied health edu- 
cation C100%, 3/3) and joint physician-nurse education 
activities (75?p, 3/4). Content classifications with the 
highest frequencies of recommendations initially rejected 
which moved to either partial or full implementation are 
scheduling and attendance of education programs (33% con- 
tinued rejection, 4/12), program design (38?o, 5/13) 
provision of hospital personnel such as DME and librarians 
for educational program development (40%, 4/10) and use of 
external educational resources (40%, 8/20). 

Under the categories increased implementation that 
is , recommendations , initially rejected or partially 
implemented, undergoing partial or full implementation 
after 18 months, no outstanding content classifications 
emerge. Generally, the recommendations listed under non- 
education recommendations have a lower increase in imple- 
mentation status (30%, 7/23) than education recommendations 
(50%, 98/197). The same holds true for the categories of 
attrition , that is, those recommendations, initially 
implemented, which decreased to partial implementation or 
were rejected. Classifications within the areas of edu- 
cation recommendations tend to be similar in attrition 
rates and the non-education recommendations tend to have 
a higher attrition rate (53%, 9/17) than education 
recommendations (25%, 45/182). 



Content Analyses of Phase II Recommendations 

During Phase II of the project, consultants su"bmitted 
295 recommendations, of which 287 were classified as having 
a primary educational objective and 17 were classified as 
being non-educational in nature. (Table 24 provides 
a summary of the content analysis of Phase II recommendations. 
A more detailed breakdown is contained in Appendix pp. 
J-3a-f ) . As j.n Phase I, consultants primarily directed their 
activities toward developing physician education programs 
in community liospitals. Of the 278 educat ion -re lat ed 
recommendations, 270 (97%) referred specifically to phy- 
sician and 3 (3%) to nursing and allied health education. 

Forty-three percent (117) of physician education 
recommendation pertained to the development of program 
resources such as financial support, educational facilities 
and personnel. The other two classifications of program 
planning and program implementation received approximately 



Table 24 



Numerical Summary of the Implementat ioa 
Status of General Categories of Phase II 
Recommendations j 
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the same number of recommendations, 80 (30^) and 73 f27^"» 
respectively, WitUn these latter two classifications 
recommendations pertaining to program design and inple- 
mentation of patient centered education programs were 
more common. 

In terms of recommendations implemented , non-education 

""""^ ^^S'^"^ full implementation rate 
1.6a., 15/17) than education recommendations (57%. 153/270"> 

irrIi:;-''%'°"/''^"'"'>' °' —education r ecommendkt ioL 
in relation to education recommendations should be taken 
into consideration when interpreting the results. It is of 

suS.;.'". ^? ' °^ ''^"^ implementation rates are 

substantially greater than Phase I implementation rates at 
the end of 4 r.onths. (Phase I: full implementation of 
non-education recommendations was 34%, 12/35; full imple- 
mentation of education recommendations was 37%, 118/3181 
As stated earlier this finding may be a reflection of the 
differences in the length of the implementation periods: 
4 months for Pr.ase I and 7 months for Phase II. Classifi- 
cations of educational recommendations more frequently fullv 
implemented are educational facilities and materials as 
prograTn resources (77%, 10/13); scheduling and attendance 
of educational programs (70% 12/J7) ; implementation of 
patient or case centered education programs (72%, 26/36)- 
and provision of house officer training (75%, 3/4). 

Two classifications of education recommendations 
^substantially higher reject ion rate: 
(82,, 14/17) regional interhospital collaboration for 
physician education program resources and nursing and 
allied health education recommendations (63%, 5/8) The 
other categories tend to cluster around the 30%-40% re- 
jection rate. 



ANALYSIS OF REPDRTED HOSPITAL CHANGES A T EXPERIMPNTAL 
AND CONTROL HQS.- I TAlT — ■ 

The second major analysis undertaken during Phase II 
T.Ll.^ deterr-.ir.ation of the differences in the nature and 
extent of reported changes between experimental and control 
hospitals. It will be recalled by the reader that this 
particular set of data pertains to hospital -initi ated 
Changes of existing hospital organization or operational 
function occurMrig during a "designated time period as 
reported by various hospital personnel to behavioral 
scientist interviewers. (Refer to pp. 116-117 for a more 
complete discussion.) 



Inspection of Table 25 permits the reader to view 
frequency of change data, both for experimental and 
control hospitals, distributed according to "degree" of 
change, that is, changes which were fully operational 
(op, category D) initiated but not yet complete ly operational 
(in, Category C § B) contemplated or tentative (co, Category A), 
and which manifested a behavioral change (be). The table 
presents the frequency and degree of overall, educational 
and non-educational changes.* 

Inferential analyses (t tests)were applied to the data 
in the total columns of each of the three classifications: 
overall, educational and non-educational, to determine the 
significance of difference between the experimental and 
control hospitals. Results of the analyses are presented 
in Table 26, p. 144 indicating that education is the only 
area in which the experimental hospitals exhibited more 
change than control hospital s to a statistical ly significant 
extent (t=1.97, p<.05). In the overall and non-educa- 
tional areas, there were no significant differences 
between the two groups of hospitals. 

Consultation specifically tried to effect change in 
the area of physician education. To rule out the possibility 
that significant results were attributable to activity in 
other education areas such as nursing, total educational 
changes were divided into sub-groups and examined to see 
if change had occurred in the area the consultants tried 
to primarily influence. A statistically significant difference 
appeared only in the physician education area (t=2.49, p<.05). 
Table 27, p. 145 presents the results of the analyses. 

Physician education changes were content analyzed into 
three reference areas roughly parallel to the classification 
of recommendations scheme, with the addition of one major 
area - physician education-related behavior. Table 28, 
p. 146 presents a content analysis of general 
categories of physician education changes. (A more detailed 
breakdown is presented in Appendix L, pp. L-2a-d.) The 
most frequent ly reported physician education changes at 
experimental and control hospitals are listed in 
Table 29, p. 147. For the most part, experimental 
hospitals showed proportionately greater incidence of 
change than control s in those areas consistent with the 



♦Educational and non - educat ion al changes were further 
classified by hospital functions, that is, personnel, equip- 
ment, physical plant, specialty areas, and communi t y . 
Appendix L, pp. L-la-b presents the frequencies of changes 
within each of these categories by degree of change. 
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ivonvcnt Analysis ot inase 1 1 
Physician Education Changes 


Experi- 
mental 


Control 


A* 


Resources 




(C ao)) 




!♦ Fiscal support including hospital, staff and other 
sources 


19 


7 




2. Educational facilities, equipment and materials 
including audio-visual aids and library 


59 


49 




3. Hospital Personnel including DME, p^. .iiologist, 
secretaries and librarian 


19 


11 




4* Outside resources including educationai. igencies 
visitinq consultants and outside soeake* 


40 


19 




Planninq and Design 







1. Program design and assessment including evaluation of 
programs, recommendations, staff needs and interests, 
and community needs; specification of educational goals; 
structuring of activities and rotating responsibility 
among staff for educational planning and implementation; 
reorganisation of committees ^ 



32 



2. Educational program scheduling and attendance, including 

preparation and publicity of prograrr.^, and keeping 
attendance and minutes of meetings „ _ . 



Programming 



43 



2. Educational use of routine hospital meetings including 
business and committee meetings 



10 



3. Content centered programs including lecture progra; s, 
discussion groups and specialty care area education 
programs, e.g . , ICU, inhalation therapy 

4» Joint physician-nurse educational activities including 

joint conferences, education pro9rar.»s built around 
specialty care ^ 

5. Miscellaneous training including self assessment test 
and postciraduate training at centers or other hospitals 

6. Interhospital sharing of educational activities and 
teaching personnel . 



18 



7. House* officer- training including resident and intern, 
g uest resident, and extern 



11 



29 



D> Behavior related' to physician Education 



1. Alteration of attendance at hospitals and professional 
meetings . • — 



25^ 



2. Alteration of attitudes or outlooks, including 
increased discussion and willingness to teach 



14 



Subtotal A. Resources 

B. Planning and Design 
C* Programming 
Behavior related to Physician Education, 



137 
41 

122 
39 



Total 



12 



((122^) 'iS2)^ 



1. Patient or case centered activities including grand 
rounds, CPC*s case presentations, and case literature 
review . 



28 



15 



10 



17 



86 
17 
82 

17 
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consultation's program development philosophy, such as 
establishment of case presentations, use of guest edu- 
cators, teaching by local staff members and establishment 
of the position of DME. 

Comparison of Hospital Attributes of 
Experimental and Control Hospitals 

m order to determine whether the statistically significant 
differ:.nce in change was due to impact of consultation, it 
was necessary to determine whether experimental and control 
hospitals differed initially (pre Phase II consultation), on 
any of the hospital or medical staff characteristics. 

The following tables and ensuing discussion deal "^^^ 
with a comparative analysis of the attributes for the 
two groups of hospitals. The entire discussion is an 
attempt to answer the question: Do any initial differences 
exist between the two groups of hospitals? Are the groups 
essentially similar and comparable or are they very 
different? 



Comparis on of Hospital A ttr ibutes of E xperiinental and 
Control Hospitals Pre Phase II Consultation 

Examination of Table 30 reveals the frequency, propor- 
tion and significance of difference bot\:ecn experimental 
and control hospitals in relation to the availability of 
facilities, personnel, and services, e.g., intensive care 
units, coronary care units, dental facilities, physical ther- 
apy departments. It is interesting to note that out of 17 
instances of services or facilities, experimental and control 
hospitals differ only in regard to the frequency of report- 
ing the existence of a physical therapy department (t=2.15, 
p<.05) and a social work department (ts2.25, p<.OS). 
In the case of both classes of facilities, experimental 
hospitals report their availability more frequently than 
the control institutions. The essential interpretation 
to be made from examination of this table is to underscore 
the likely equivalence of the two groups of hospitals in 
regard to the existence of most kinds of service facilities 
and resources . 

Table 31 describes a series of attributes or charac- 
teristics of both control and experimental hospitals re- 
lating to continuing education (e.g., existence of a hospi- 
tal-salaried DME, existence of an education committee). It 
should be noted that in some 25 instances of comparison. 



CQNPAKISON (t TE:»T) OF EXnUIMENTAL AND CONTROL HOSTITALS ON I 'OSriTAt CHAXACTmSTICS 

YIELOINC DISCONTINUOUS (OISCRbtE) DATA 







Exper iacnt a 1 






Control 






Hospital 
Characteristic 

Existing Facllili«l 
and Services 


Yes 


N -C ase s 






Yes 


N*Cas«s 


n 
X 




t_J^^a l_ue 
(sTgrTi f* icanct 
of difference 
betwaen pro- 
portions) 


ICU 


22 


31 


.SI 


.08 


19 


37 


.SI 


.08 


.61 


ecu 


19 


31 


.so 


.08 


19 


37 


.SI 


• .08 


.09 


Orfanised Lner^ency 
Care Departncnt 


31 


33 


. 94 


.04 


33 


36 


.92 


OS 


.32 


Organized Outpatient 
Oepartncn t 


23 


37 


.62 


. OS 


1 7 


37 


.46 


. 08 


1 T ft 


Clectroencephalof /^phy 


11 


37 


.30 


.07 


12 


36 


.33 


.08 


.28 


Hone Care Program 


7 


31 


.11 


.06 


9 


37 


.24 


.07 


.64 


Progressive Patiant 
Care Service 


4 


26 


.IS 


.07 


3 


29 


.to 


.06 


.S6 


Tost -operat ive Recover) 
Room 


36 


38 


.ds 


.04 


31 


37 


.84 


.06 


I.S6 


Inhalation Therapy 
Depart nent 


34 


3S 


.63 


.OS 


20 


36 


.S6 


.08 


.61 


Patholocy L:»6 (w/path» 
OlOl'.l s t ) 


33 


38 


.87 


.OS 


:8 


37 


.76 


.07 


: .22 


Dentttl racil.ties 


IS 


32 


.47 


.09 


IS 


36 


.42 


.08 




riiaroticv (i» / rc ^ i s t erca 
«'.)iar.4;sacist ) 


33 


37 




.05 




37 


.81 


.06 


s i 


Physical Thcrwpy Dept. 


32 


38 




.06 


23 


37 


.62 


.OS 




Premature Nursery 


23 


36 


.64 


.08 


22 


3S 


.63 


.08 




Psychiatric {n^Patient 
Unit 


4 


36 


.11 


.OS 


1 


36 


.03 


.03 


1 


Rehabilitation In*, 
pat lent Un 1 1 


7 


36 


. 19 


.07 


2 


36 


.06 


.04 


1.68' 


Social l^crk Pcpt . 


20 


36 


.S6 


.08 


11 


37 


.39 


.07 


2.:s** 


Adain ist rator Charac- 
ter! Stic 




















Fomal Dc}:re« in Hos- 
pital AJaini^tration 


11 


38 


.29 


.07 


10 


36 


.28 


.08 


.11 


Participation *n Con- 
tinuing EJucation Pro- 
grass wtthin Past Yr. 
Relative to Role as 
Ho s *) 1 1 J ! A ! : r. * t r 1 • 0 r 


30 


37 


81 


.06 


?3 


33 


. -6 


0- 





• P i . 10 
•• pi. OS 

t Scrutiny of the data indicated that thera was no relationship between tha dependent 

and indepejident variables. Therefore, a statistical computation (x^) was not pexforaed. 



TULDINC OlSCCNTiSUOUS (OlSCRCTE) DATA 







Expcrl^efltal 








CoiOfoJI^ 












r 








Sp 


t V«il*it» 


tor c • *^ * * ^ ■* * * 




















Ho$p. -s^lar i«<l Dir. of 
Medical ucaticn 




U 




.07 


4 


37 


.11 


.OS 


1 .S6 


education Ccr.nitt^e 




37 


.6S 


.Oft 


19 


3? 


.SI 


.Oft 


1 .49 


C»»iiittec or Director 
of llosp.*waJc Ed. ( 
Traininc rrei:ras 


9 


3« 


.24 


.07 


S 


34 


-IS 


.06 


ft 


Kolicitatiou of A»$a$. 
1 .^nKft f ri*M > . 1. ^ M«a . 

. 1 .1 . .. 


1 


M 


• 

1.* 




; 


1 1 


Ml« 


IM 




1*1*1 t 1 } 

U U ». J I * <« H 1 ' M« f '>J JJl 




















AppruvvM f<»««i. ^Cm«^«>« 
Affiliatien 


4 


3t 


.11 


.OS 


2 


37 


.OS 


.04 




ftcsiJcncy U Inter::ship 
Pron^raa 




3t 


.24 


.07 


6 


37 


■ 16 


.06 


s 


Ijktcrn or Sur.acr 
^ t u Ji-n t 1* r n 4 r a** » 


13 


32 


.41 


.01) 


6 


32 


.J9_ 


.07^ 


1.9}* 


Trofessional Nursing 

"icftool 


:c 


3S 


.26 


.07 


ft 


37 


.22 


.07 


t 


Affiliated Nurs;n; 
S t u dent $ 


1 1 


30 


.37 


.09 


9 


34 


.26 


.Oft 


.9S 


Explicit Orientate o n 
( Education Proiras 
for Truaiccs 


14 


36 


.39 


.OS 


6 


36 


. 17 


.06 


2. 10»» 


Interho»pi ; A 1 U..:ation 
^ro^rans (coi la;; ; rat ion 
in planning or exc^atioi 

kith anotl*. hocitftl) 

rhva i c ; an 


11 


3i 


.31 


.Oft 


12 


31 


.39 


.09 


.6ft 


Nurs I nc 


2it 


36 


.72 


.Oft 


2S 


33 


«76 


.Oft 


.33 


Allien IICAiK'i 


J 3 


32 


.41 


. 09 


16 


31 


.S2 


. 09 


. ft7 


Exist cncc of Conrittees 




















^ m m M m ^ i» % A V** 


SS 


3t 


1 «00 




36 


36 


1.00 




t 


Internal *:ciiical *«dit 


:i 


36 


.St 


.OS 


19 


37 


.SI 


.Oft 




External ^'eaUal ^udit, 
i.e., rAS-."\? 


13 


3t 


.34 


.Oft 


10 


37 


.27 


.07 


t 


Functioning Trustee* 
Medical St at'i* - a; • 
stration Co*-3.t:ee 




3t 


.•4 


.06 


33 


3ft 


.92 


.04 


1.07 


Use of Lxtrar.ural Ccr.sul* 
tants 




















:«urs inc 


:o 


3S 


.57 


.Oft 


14 


34 


.41 


.Oft 


1.33 


i!o$?ital Adr.inis; ration 


13 


36 


.36 


.CS 


A 


' 30 


.27 


.Oft 


t 


Allied Mealt:. 


7 


33 


.21 


.07 


7 


' 31 


.23 


.07 


t 






• r 








t 

t T 






! 90 •• 1 



•Pi-.*o 
••f i;.os 

t Scrutiny of ".r^t data indicated th»t there w^s no relationship between the dependent and 

indeperdent variables. Therefore, a statistical computation (x^) was not performed* 



Only two instances in which a significant difference 
arpear are in regard to the provision of orientation 
and education programs for hospital trustees (t=2.10, 
p<.05) and the use of extramural consultants by phy- 
sicians (t=1.99, p<.05). In this regard, experimental 
hospitals have a much more frequent utilization of training 
programs and outside consultants in contract to the con- • 
trol hospitals. Again, the essential implication of the 
body of data is to underscore the comparab ility of hos- 
j^'itals in the exper im<^ntal and control groups respectively. 
It is of incidental notice that the existence of hospital 
salaried DME's is relatively small (9 and 4 in the exper- 
imental and control groups respectively) . Similarly, 
the existence of a committee or Director of Hospital-Wide 
Education and Training is a quite infrequent device for 
the administrative supervision of in-house education. 
Similarly, it should be noted that both the experimental 
and control hospital groups are unlikely to have approved 
medical school affiliations, to have residency and intern- 
ship programs, extern or summer school programs, etc. In 
a similar vein, the relatively infrequent existence of an 
external medical audit is noteworthy as is the extremely 
low frequency of the utilization of extramural consultants 
in regard to allied health programs. 

Examination of Table 32 again reportirg additional 
dimensions of hospital attributes for the two groups of 
hospitals, reveals essentially equivalent entries for the 
two groups of hospitals in regard to the number of beds, 
ratio of active staff to beds, aggregate mileage index 
reflecting distance from nedicaJ schools, total budget 
percentage allocated to physician education in contrast 
to allied health education, turnover rate of staff, etc. 
Again it is noteworthy to a*^ know ledge the fact that ex- 
perimental and control hospitals show no apparent sta- 
tistically significant differences in attributes. 

Comparability of Medical Staff Characteristics Between 
Experimental and Control Hospitals 

Examination of Table 33, P,153 permits one to compare the at- 
tributes of the medical staff along a numoer of dimensions 
between experimental and control hospitals. Results of 
the analysis indicate that there arc no statistically 
significant differences between the two groups of hospitals 
in regard to such attributes as size of active medical 
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Table 33 155 
COMPARISON (t TEST) OF EXPERIMENTAL AND CONTROL HOSPITALS 

ON MEDICAL STAFF CHARACTERISTICS YIELDING CONTINUOUS DATA 







Experir 


cntai 




Control 




t Value 


Mcdic^; Staff 
















Ch^rACterist i c 


X 


S.D. 


N 


S.D* 


N 


(Signi f icanc6 












of difference 
between X's) 


Ccv.rtesv Staff Char .c- 
teristics 
















Size of Courtesy Medi- 
cal Staff (#) 


21.8 


28.7 




20.78 


23.3 


36 


.16 


Ae-ive Staff Chaiacter- 
istxcs 
















Size of Active Medical 
Staff {^) 


42* 02 




34 


31.08 


27.5 


37 


1.12 


y.ean age of staff 


48 


5.7 


35 


47.96 


4. 3 


3C 


.03 


Staff w/other hospi- 
tal affiliation i%) 


50«48 


34.6 

1 


31 


53. 57 


38.9 


32 


.33 


cenerax riact* t 

on Active Staff (%) 


37.3 


21.1 • 


30 


48.28 


24.3 


30 


1.83* 


Board certified C^) 


35.8 


21.4 


32 


33. 56 


19.7 


35 


.44 


Keiabership in Satl* Hon 


46* 


28.3 


27 


44.54 


28 


29 


.19 


Willingness to attend 
teaching sessic.*s (%) 


60«5 


19.2 


33 


75. 96 


28.3 


26 


.72 


willingness to Instruct 
i%) 


59.6 


29 


32 


54.05 


31.9 


26 


.68 


willingness to use own 
patients in education 
programs (14) 


81.4 


21.6 


28 


71. 96 


33.8 


24 


1. 19 


willingness to be taught 
by oitfn hospital staff (>^) 

- 


75.5 


30.1 


31 


68.15 


33.5 


19 


.79 


3* rector cf VediCBkl Ed. * 
C-aractor.s^ics 
















Age 


At 14 


13.20 


28 


50.6 


12.62 


5 


.53 


Length of Service in 
Current pcsition 


2.8 


3.6 


25 


2.33 


1.4 


6 


.32 


C"-ief of y.^dicir.e Cnar- 
acv eristics 

Age 


51.7 


7.5 


24 


47.4 


7.6 


18 


1.75 • 


Length of Service in 
Current position 


5.77 


4. 57 


23 


8*26 


7.48 


19 


1.29 


?-ief of S'Jtcerv Char- 
acteristics 
















Age 


49.2 


6.85 


25 


50.11 


7.77 


19 


.40 


Length of Service in 
1 Currc'.t Positicn 

t_ 


4,96 


4.14 


25 


5. 62 


S.88 


18 


.42 



• f <.io 



staff, mean age of staff, number of board certificw members 
of staff, membership in national honorary academies, and 
interestingly enough, the reported willingness on the part 
of staff to attend and/or instruct teaching sessions. 
Examination of Table 34 permits one to compare quickly the 
exi stence of f ul 1 time radiologists, pathologists , chiefs 
of medicine and surgery, and/or directors of medical educa- 
tion. It is interesting to note that the only item that 
approximates a statistical ly significant difference between 
control and experimental hospitals is the item of 
participation of the DME in continuing medical education 
programs in the past year (t=2.29, p<.05). 



ANALYSIS OF FACTORS INHIBITING OR FACILITATI N G THE 
IMPACT OF CONSULTATIOy 

Having demonstrated the lack of divergence between 
manycharacteris tics of the two groups (experimental 
and control hospitals), it is tenable to assume that 
the hospitals tend to be similar to begin with, and an 
investigation dealing with the effect of a characteristic 
on change measures will have comparable effect (be due to 
consultation rather than the nature of the hospital), since ^ 
it can be assuned, we are comparing the effect of two similar 
things . 

In accordance with the corollary research objective 
of the present investigation, that is, determination of 
factors facilitating and/or inhibiting the impact of 
consultation, the research question posed was of the 
following nature: What is the effect of the presence 
of selected hospital, medical staff, and consultant 
character istics on various dependent change measures 
(that is, hospital changes and recommendation implemen- 
tation rates) . 
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Analysis foHoKed a format similar to that of Phase 
I, howQVer with modification and rcKorkinq of both data 
collection and analysis . A more thPrOugli and extensive 
analysis of possible facilitating and inhibiting t actors 
was undertaker, during Phase II and will be presented in 
the present section. 

The reader will recall that Phase I analysis called 
for utilization of the following dependent meai^ures: 
1) full impler.entation of recommendat ion rates; and 2) 
full plus partial implementation of recommendation rates 
Time was not sufficient to use a third dependent measure 
that is, the r.unber of reported education changes. Ela- 
borating on Phase I, Phase II research yielded a wealth 
of dependent measures, due to both the indepth reporting 
of hospital change data and the additional collection 
of data pertinent to the implementation status of Phase 
I recommendations after 18 months. 

Change measures yielded for Phase II were as follows 

1. Recommendation Data (total, educational, non- 

educational) 

a. Phase II recommendations: 

Fully implemented ( I ) 

Partially implemented (P) 

fully plus partially implemented (IPI) 

Jve j ec t ed (R) 

b . Phase I recommendat ions after 18 months 

f u 1 ly implemented ( I ) 

Partially implemented (P) 

Fully plus partially implemented (IPI) 

Rej ected (R) 

2 . Hospital Change Data 

a. Areas of change : 

Total changes 
Education changes 

Physician 

Nursing 

Allied health and others 
Non-education changes 

b. Degree of change: 

Operational (Op) 



Initiated but not yet complete (Ino) 
Contemplated CCo) 
Behavioral CBe) 

Since it would have proved unfeasible, impractical 
and time consuming to utilize every dependent measure 
in inferential analysis, the project staff was beset 
with the task of determining which measures to select. 
An intercorrelat ion matrix (Pearson r correlation 
coefficient) was computed for ALL Phase I and Phase II 
measures (52 variables yielding 1479 correlations). The 
major correlations are presented in Tables 35, 36, 37, 
and 38. The following variables were selected to be 
utilized to test the relationship of the occurrence of 
change to various independent variables: 

1. Phase II full implementation plus partial 
implementation of recommendations (hereto- 
fore referred to as Phase II IPI.) 

2. Phase I full implementation plus partial 
imp leinentat ion of recommend at ions at the 
end of 18 months (Phase delayed IPI) 

3. Phase II reported hospital education changes 
(not broken down by degree of change) 

Variables in items 1. and 2. were broken into frequency 
of combined implementation (number) and proportion of 
frequency of combined imp leinentat ion to total number of 
recommendations submitted to the hospitals (percent) . 
The variable in item 3. was broken into frequency -of 
occurrence (number) and proportion of number of education 
changes to total number of changes (percent). A variety 
of tests were used, x2, r, ANOVA, depending on the nature 
of the data and the question being answered. 



The Interrelationship of Sclected^ Hospital 
Characteristics and Occurrcnce^^ Change 

Efforts at ascertaining the possible iiteraction ef- 
fect of selected attribute's of hospitals and three classes 
of dependent measures, i.e., number and percent of partial 
and completely imolement <=»H recommendations at the end of 
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Phas e II and the numb er and percent of changes report ed 
having occurred in the hospital pertinent to education , 
may be examined in Table 39. Medians were determined for 
each of the three dependent variables and X^'s were 
computed for the hospital characteristics yielding 
dis continuous data according to whether the dependent 
variable fell above or bt3low the median. It is important 
to note that a substantial number of attributes do not re- 
late to the dependent measures used in the current study, 
©•g** the presence of a large number of special services 
and facilities such as an outpatient department, electro- 
encephalography, a progressive patient care program, an 
inhalation therapy department, a dental facility, a phar- 
macy, a physical therapy department , a premature nursery, 
a psychiatric in-patient unit and a rehabilitation in-patient 
unit. Similarly, the existence of an int erhospital physician 
or allied healt!. education programs do not s eem to 
be associated with differential implementation of recom- 
menuations or occurrence of hospital education changes. 

Findings bordering on statistical signi f icance and 
findings r each i n g s tatj^s tical signi f icance tend to suggest 
that the presence of an intensive care unit ana piivTiary care 
unit, the presence of a hospital -salaried DMH , the presence 
of an education committee, the existence of a committee or 
director of hospital-wide education and training programs, 
the existence of a pathology lab and the presence of a so- 
cial work deparinent seem to be associated with differential 
implementation of recommendations or the occurrence of edu- 
cation change. Another way of stating this is that other 
things being equal, hospitals, selected to the attributes 
such as those listed immediately above, would be more likely 
to benefit fror. education consultation programs such as 
described by the current project than hospitals vhich do 
not have such attributes. 

Examination of Table 40 reveals an additional effort to 
explore the relationship between selected independent va- 
riables, e.g., attributes of the hospital and/or its se- 
lected members of its supervisory staff and the three de- 
pendent measures: n amber and percent of partial and com- 
pletely implemented Phase II recommendations, number and 
percent of partial and completely impl emented Phase I re- 
commendations at the end of 18 months, and the number and 



Tabu 39 • 

RELATIONSHIP (X^) OF HOSPITAL EDUCATION CHANCES AND RECOMMENDATIONS 
TO PRESENCE OF HOSPITAL CHARACTERISTICS YIELDING DISCONTINUOUS (DISCRETE) DATA 

IN EXPERIMENTAL HOSPITALS 



U3 



IndepcnUent Variable 






Dependent 


Variables 


















Hospital Characteristic 


Phase 


II in 


Phase I Delayed 1PI 


Hosp. Ed. 


Changes 




Number 


Percent 


Number 


Percent 


Number 


Percent 


Ei^istinf Facilities ( Services 














ICU 


1.03 


1.78 


4.23** 


4.26** 


.87 


.43 


ecu 


3.31* 


.90 


.04 


1.63 


3.80** 


.11 


Org. Outpatient Dept. 


.29 


.00 


.43 


.43 


2.31 


.00 


Electroencephalography 


± 


t 


t 




1 


1 


Home Care Program 


.01 


.59 


♦ 


1 


2.30 


2.80* 


Progressive Patient Care 


t 


t 




1 


1 


t 


Post -Operative Recovery Room 


tt 


tt 




11 


11 


11 


Inhalation Therapy Dept. 


.46 


.91 


1.46 


.61 


4 


1 


Pathology Lab (w/pathologist) 


.00 


.00 


3.49* 


1.43 


11 


It 


Dental Facilities 


i 


4 


1 


4 


1 


.1 


Pharmacy (w/rcgis tered pharmacist) 


♦ 


4 


1 


± 


4 


1 


Physical Therapy Dept. 


.13 


.13 


f 


t 


11 


11 


Premature Nursery 


t 


4 


1 


t 


1 


1 


Psychiatric In-patient Unit 


4 


4 


1 


t 


4 


1 


Rehabilitation ln*patient Unit 


no 


no 


.81 


.06 


. 03 


.71 


dOClcA woris vcpw . 


4 4 


tt 


.1 


3.9S** 


4 4 


11 




4 ^ 


4 4 


11 


±± 


11 


11 


Internal Medical Audit 




tt 


11 


tt 


11 


It 


cxternai pieuicoi nuuiwi 
i.e., PAS-MAP 


• 27 


1 .08 


.24 


.55 


.63 


- — — — 1 


Use of Extramural Consultants 














Nursing 


tt 


tt 


4 4 


♦ 4 


11 


11 


Hospital Administration 


tt 


4 4 


4 4 


11 


11 


11 


Allied Health 

• 


tt 




11 


4 4 




z z 


Physician 


.01 


2.52 


2.42 


.88 


.69 


1.54 


Administrator Characteristics 














Formal Degree in Hosp. Admin. 


1.19 


1.85 


.08 


.51 


.01 


.00 


Participation in Continuing 
EduCi'>tion Programs within Past 
Year Relative to Role as 
Hospital Adniiust Tutor 


± 


4 


.58 


.0 


.01 


• 01 


• n ^ 10 









p t -OS 
p * .01 

> 1 Scrutiny of the data indicated that there was no relationship between the independent and 

^ dependent variables. Therefore, a statistical computation (X^) was not pc:'formed. 

„lS!r^ ^ Data was insufficient to perform computation (X^) . 
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CONTINUATION OF TABLE 39 
RELATIONSHIP (X^) OF HOSPITAL EDUCATION CtiANCLS AND RICOMMENOATIONS 
TO PRESENCE OF HOSPITAL CHARACTERISTICS YIELDING DISCONTINUOUS (DISCRETE) DATA 

IN r;ypC»,!MrNTAL lIOSriTALS 



1 ; 

} 1 Independent Varinblt 


Dependent 


Variable 




! ^ Hospital Characteristic 


1 Phase 


II ri 


Pha5c I Delayed IPI 


Hospit al 


Hd. Chan^. V 




' Number 


Percent 


Number 


Percent 


Number 


Percent 


■ 

1.1 


Arrangements for Continuing Educati 














1 Presence of Hospital-salaried 
) 1 Director of Medical Education 




2.89* 


.27 


*09 


1.3S 


.04 


i 1 

1 ' Existing Education Conaittee 


1.19 


1 .76 


S.93 


• • 

4.62 


3.98 


1.67 


^ ^ Functioning Trustee*Med. 
/ 1 Staff-Adm. Committee 


It 


tt 


.24 


.SS 


.09 


.1.78 


1 i 

ConiRittee or Director of Hos- 
pital-wide Education ( Training 
1 P/ogra» 


3.13* 


2.24 


.2S 


3.04* 


4.46" 


9.32*** 


Solicitation of Assistance fron 
N.E. Mod. Schls. in Dovelopment 
1 or liipleiaentat ion of P.iysician 
• 1 Education Programs 


tt 


tt 

• 


t± 


tt 




tt 


! Existence of Formal Education Programs 














1 


Approved Medical School Affilia- 
tion 


t± 


zz 


♦ ♦ 


tt 


tt 


tt 


Residency ( Internship Program 


2. OS 


.31 


.2S 


.86 


2.33 


.00 


1 Extern or Summer Student Program 


1 


t 


♦ 




t 


t 


Professional .Vursing School 


.11 


.31 


.01 


.21 


.03 


.S4 


< 1 Affiliate Nursing Students 


1 


t 


♦ 


t 


t 


1 




Explicit Orientation ( Education 
Procram for trustcas 


.02 


2.2i 


1.S2 


2.20 


.ir 


3.05* 




Interhospital Educatinn Programs 
(col laboratiok. in planning or 
oxecution with another hospital) 
















Physician Education 


.33 


1.47 


.14 


189 


.25 


.S2 




Nursing Education 


.•7 


2.73* 


.00 


1.66 


.62 


.03 


i 


Allied Health Education 


2.29 


1.17 


1.22 


.62 


.43 


.00 

r — !i 



• P * .10 
P ^.05 
P * .01 

t Scrutiny of the dulA indicated that there was no relationship between the independenv 
and dependent variables. Therefore* a statistical computation (x^) was not perf'^rmed. 
11 Data was insufficient to perform cocputation (x^) . 
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percent of changes reported as taking place in the 
hospital pertaining to education. A Pearson r corre- 
lation coefficient was used to determine the correlation 
between the independent and dependent variables. A num- 
ber of hospital characteristics significantly correlated 
with the three depen,.ent variables, e.g., a positive 
correlation between the number of beds and the number of 
Phase I recommendations-implemented as w^ll as with the 
number of reported hospital education changes . A nega - 
tive correlation was yielded for number of beds and per- 
cent of hospital education changes. This may seem para- 
doxical in light of the positive correlation with number 
of educat i on changes . However , no relationship neces- 
sarily exists between number and percent of changes as 
number is the total number of education changes occurring 
and percent is the proportion of educational changes to 
totalchanges occurring. Other significant negative 
correlations were: number and percent of Phase II 
recommendations and total miles hospitals were located 
away from medical schools, indicating that the closer 
to medical schools the higher the implementation rate; 
medical staff turnover rates and percent of education 
changes ; nursing staff turnover rates and Phase 1 1 imple- 
mentation of recommendations; age of administrator and 
number of hospital education changes; and administrative 
experience of di ector of nurs ing and percent of Phase 
II recommendations implemented. Positive correlations 
included percent of budget allocated to physician educa- 
tion and number, ofe.^education changes; percent of budget 
allocated to aMied health education and number of edu- 
cation changes; and /age of director of nurs ing and per- 
cent of educations-changes. 

The significant correlations mentioned above should 
not be interpreted as more than perhaps suggested find- 
ings . However, extremely cautious interpretations of 
these significant correlations would suggest the follow- 
ing: 

1. Other things being equal, select hospitals of 
larger number of beds as potential candidates 
for being responsive to an educational consul- 
tation; 

2. Select hospitals which are closer to medical 
centers as potential responsive candidates for 
educational consu Itat ion ; 
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Table 40 

RELATIONSHIP (CORRELATION - PEARSON r) OF HOSPITAL EDUCATION CHANCES 
AND RECOMMENDATIONS TO HOSPITAL CH ARACTLRISTICS 
YIELDING CONTINUOUS DATA IS EXPERIMENTAL HOSPITALS 



Dependent Variables 



Hospital Characteristic 
NuBber of Beds 



Phase II iri_ 

r « ♦.13 r « ♦ .02 
n « 27 n « 27 



Ratio of Active Staff to 
NuBber of beds 



r = ♦.20 r 

n « 27 n « 27 



18 



Total Miles froa Medical 
Schools 



n = 26 



41»» r « -.61* 
n « 26 




Total Budget (1967-6S) 

% Budget f/Physician 
Education 

% Budget f /Allied Health 
Educat ion 



Turnover Rates (•) 
Medical Staff 



Nursing Staff 
Board of Trustees 



r 
n 



-.28 
16 

-.11 
IS 



♦ .27 
16 

♦ .13 
IS 



r = .00 
n » 21 



r = ♦.OS 
n = 21 



r = ..SO** r = ,00 
n =: 19 n * 19 



♦ .j: 

22 



r « ^.04 
n * 22 



r B ^.02 

n = 33 



r = ♦.It 
n * 20 

r = ♦.34 
n = 18 



r = -.21 
n = 53 



r » ^.14 r « ♦21 

» 37 n » 37 



♦ .01 

20 



.68^ 



24 



r « ♦.04 

n >^ 18 



r e ♦.SS* 

n « 22 



♦ .34 
24 

♦ .12 

22 



r 




-.19 


r 


s 


-.1 


n 




28 


n 




2S 


r 




♦ .28 


r 


s 


♦ . 4 


n 




22 


n 




22 


r 




.IS 


r 


E 


♦ . 


n 




26 


n 


S 


26 



r = -.28 
n = 51 

r = ^.07 

n * 26 

r • - OS 

n = 31 



. -.44** 

* 31 

= ^.12 
« 26 

= ♦.ll 
> 31 



Administrator Characteristics 
Age 

Years in Administration 
* Years in Current Position 




Director of Nursing Character 
istics 

Age 

Years in Administration 
Years in Current Position 



-.07 
27 



r « 12 
n 26 

r = ♦ .03 

n = 27 



r = ^.13 

n - 27 

r = -.09 
n » 26 



r 


s 


-.27 


r 


s 


-.33 


r 




♦ .IS 


r 


s 


n 




21 


n 


s 


21 


n 


s 


24 


n 


s_ 


r 


s 


♦ .09 


r 


a 


-.39* 


r 


3 


♦ .11 


r 


3 


n 


s 


22 


n 


s 


22 


n 




24 


n 


8 


r 




♦ .18 


r 




-.15 


r 


S 


♦ .13 


r 


K 


n 


K 


23 


n 


s 


23 


n 




26 


n 


B 



.02 



52 



.07 



32 



r ^ *.22 
n = 32 

r = -.06 
n = 32 



24 

-.11 
24 

-.23 
26 



r « ..38** r 

n « 37 n 



r » -.16 
n = 37 



-.26 
37 

-.24 

37 



r = ^.21 
n « 30 

r = ♦U 
n = 29 

r = ♦*09 
n ' 31 



r = ♦31 
n ' 30 

r a ^.02 
n « 29 



-.07 
31 



• P ^.10 
P ^ .05 
p 5: .01 



3. Select hospitals which allocate a greater per 
cent of their budgets for physician and allied 
health education; 

4, Select hospitals with lower turnover rates of 
nursing and physician staffs as potentially re- 
sponsive candidates for education consult at ion • 

*S. Do not be concerned with the ratio of active 
staff to number of beds; the number of years 
the administrator has been in his current posi- 
tion or years in hospital administration; nor 
the nuiaber of years the director of nursing has 
been in her current pos i t ion . 



Interrelationship of Selected Medical Staff 
Characteristics and Occurrence of Change 

In an effor*" ascertain an answer to the question, 
"Is the existence of certain medical staff character- 
istics, e.g., size and composition of active medical 
staff, or the presence of a director of medical educa- 
tionorchief of service associated with the increased 
occurrence of education change and/or implementation of 
recommendations?", a series of analyses were undertaken, 
i.e., significance of Pearson r's or x^*s between pre- 
sence or absence of the implicated professional medical 
staff and/or one of the three classes of dependent mea- 
sures - number and percent of partial and completely 
implemented recommendations , and number and percent of 
education changes reported having occurred in the hos- 
pital . 

With reference to Table 41 , very few significant 
correlations were yielded between medical staff charac- 
teristics and the three dependent variables. Of note 
are the following significant correlations: size of 
active medical staff to number of hospital education 
changes indicating that the larger the size of the ac- 
tive medical staff the greater the number of education 
changes (r = +.31); the fewer percent of board certified 
on the active medical staff the greater the number of 
Phase II recommendations implemented (r = -.38); the 
greater percent of staff with membership in national 
honorary academies the greater the number of Phase II 
recommendations implemented (r = +.44); the willingness 
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of the medical staff to either participate as instructor 
in the hospital -based CME program or willingness to be 
taught by other members of the hospital medical staff, 
the fewer the number of Phase II recommendations imple- 
mented (r = -.46; r = -.70 respectively); the older the 
DME the greater the number of education changes (r = 
+.46); the longer the DME has been in his current posi- 
tion the scalier the percent of education changes; and the 
older the chief of medicine the greater the number of 
Phase II recommendations implemented. It is of interest 
to note that none of the medical staff characteristics 
correlated with percent of recommendations implemented. 

Examination of Table 42reveals that in general, the 
presence of radiologist, pathologist, chief of medicine, 
chief of surgery, are not associated with any unusual 
degree of implementation or recommendations and/or edu- 
cation changes. However, the presence of a full-time 
radiologist, pathologist and pharmacist each appears to 
be related in a statistically-signi'^icant way to one or 
more of the dependent measures. 



The Interrelationship of Consultation 

Characteristics and Occurrence of Change 

In an attempt to ascertain the relationship between 
various consultation characteristics and two change 
measures of Phase II continuing medical educati.n growth 
(i.e. Phase II IPI and hospital education changes) , a 
series of additional anal>ses were undertaken. 

Phase II consultation characteristics represented a 
complete revision ^'er Phase I characteristics utilized. 
Since present data wOllected was relevant to Phase II 
consultants only, use of "Phase I delayed" as a dependent 
measure was irrelevant to present analyses. 

Phase II consultation characteristics dealt with 
two areas: 

1 . Consultation process characteristics 

Data distributed according to hospital: 

a. Number of training sessions attended by the 
DME. 
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b. Same or different consultant for Phase I 
and Phase II. 

c. Interval between transmittal of recommen- 
dations and termination date of "implemen- 
tation period" (date of interview). 



2. Consultant characteristics 

Data distributed according to consultant. 

Since one consultant visited, on the average, 3 
hospitals, it was necessary to accoimiodate the 
dependent variables of each of his hospitals to 
produce an average change measure for the con- 
sultant. 



The following consultant characteristics were 
utilized: 



a. Number of training sessions attended by 
consultant . 

b. Age of consultant 

c. X number of visits made hy consultant to 
his hospitals. 

d. Membership of consultant in specialty 
(scientific) societies . 

Inferential analyses (t tests of significance be- 
tween X change rates) , conducted on the data depicted in 
Table 43 yielded the following results. The reader will 
note that the only consultation characteristics which 
appear to differentiate between a high and low change 
rate are; 



1. Visits to hospitals by consultants 

Hospitals of those consultants who made, on the 
average, 2 or more consultation visits, imple- 
mented a significantly greater percentage of the 
recommendations transmitted to them when com- 
pared w5.th the hospitals of those consultants 
who made less than 2 consultation visits. 
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2 • Consultation visit by same or different consul - 
tant during Phase I and Phase II 

Hospitals who experienced the same consultant 
during both phases of t^he project reported a 
significantly greater X number of hospital edu- 
cation changes (X = 22 •SG) when^compared with 
those hospitals who-experienced visits from 2 
different consultants (X = r4.30) during the 
two different consultants {\ - 14.30) during .he. 
two different phases of the project. 

All in all, the above analyses attempting to relate 
differential attributes of the hospital, medical staff, and 
consultation to differential rates of implementation of 
recommendations and/or changes pertaining to education 
are at best suggestive rather than definitive. The pri - 
mary lack of definitiveness of these findings can be 
readily attributed to the minimal sufficiencies of num- 
bers of cases involved in the analysis and, in some in- 
stances, to the very large variability reported in con- 
nection with particular selected attributes. 

Interaction Between Hospital S Medical Staff 
and the Receiving of Edu c ational Consultation on 
the Frequency of Euucat ion-al Changes 

In this unit a series of analyses attempting to assess 
the combined effects of hospital attribute (i.e., activity, 
service, etc.) and the presence of the independent variable 
(i.e., the presence of education consultation) will be 
assessed in terms of a greater or lesser apparentness of 
educational changes reported in the hospitals. In these 
instances two way analyses of variance have been performed 
on a variety of hospital attributes data (Table 44, pp. 
174-180). It should be noted that the analyses have been 
done on both the number of education changes recorded by 
the hospital and the proportion of education changes to 
total number of reported changes. This replicati-wi — / 
involving two dependent measures was undertaken to note 
the consistency of findings when the possibility exists 
for certain biasing effects in the recorded changes. (For 
instance, the interview used- in one hospital might have 
recorded solely educational changes in contrast to a second 
hospital which might have recorded many more types of 
changes in which the educational changes were but a 
relatively smal 1 portion . Furthermore , some hospitals 
because of size, etc., might have had relatively few to 
report, whereas other hospitals might have had substantial 
numbers to report and yet the proportion of educational 
changes might have been relatively comparable between the 
hospitals • 



J 



CO 

H 
CO 

M 

H 

< 

< 
s 

< 

M 
0. 

CO 

o 
o 



H 
CJ 

Ui 

(U 
X 
H 

DU 

o 
o 

CO 
M 

< 

Z 

o 
u 



0) 
< 

f-i 
iO. 

§- 

•J 

o 

H 

o 
u 



CO 

> 

•J 
< 

H 
X 



4J 
O 

o 

«H 
Ol 

c 
o 

o 
it 



u 

r-l ♦> 

•H M 
M 4) 

> u 
(d 
• u 

C US 



O c« 

<H O 

a: 

ua 



mi 



CO 

o 
< 

X 

u 
o 

HI 
H 

< 

u 

:d 

a' 

va- 

< 
H 
HI 
Ou 
CO 
O 
X 

% 
O 



4> 

C /-> 
4) 

6 O 
€> C 



O O 

O p. 

O K 



41: 



(0 

o 

(0 

O 

4^ 

a 

M 



ERIC 



i 



CO 
2 



i 

U« 



10 
O 
• 

CM 

n 
u 



CO 



CO 



in 
o 

• 

o 
f-l 

i 

u 



i 



II i 



to 

2; 



in 
o 

i i 

n 



o 



CO 



i 



in 

01 



u 



CO 
2 



CO 



o 
in 

i 

M 



in 
o 



00 

o 



in 
o 



•0 




c 




cd 




(0 




4) 




*H 








*H 




•-4 




*H 








Cd 






















10 


c 


4) 


0 




4^ 




tf) 


> 


•H 


u 




4) 


W 


CO 



in 

i I 



O M4 



CO 

2 



i 



CO 



i 

U« 

f-l 

o 

H 



CO 
2 



CO 
2 



CO 
2 



O 



tn 
o 



i 



00 

o 



in 
o 



in 
n H 



0 



o 

a 

c 
o 

Cd 
o 



60 

o 



CM 

o 



i 



CM 



i 



CM 

M 

M 



CO 



00 

o 



i 



in 
o 



i 

pu 




CO 
2 



CO 
2 



in 
00 

i- 



i 

pu 



to 

i i 



CO 



in 

a% o 



i i 



i 

Ou 



p« 
cd 

M 
60 
O 
•-4 

Cd 
P« 

4) 

a 
c 

o , 
o 

M 
♦* 

a 

41 

rH 

Ul 



o 

CM 

i 



CM 
H 
U 



CO 
2 



CO 

• 



00 
CM 



U 



o 
o 



I 

Ou 



tn 

O 

O 

^ |*-4 O 
r-l iHlr^ • 



I i 

Pu «0 



i 

Ou 



a 
cd 
u 

60 
O 
H 

o 

H 

cd 
u 

o 

e 
o 

X 



in 
in 



i 

UU 



EL. 



CO 



CO 

7: 



in 

CM 



CO 
2 



tn 




0 






1 ^ 








•-4 0 


'•^ 1 






r • 


Im 




i i 


i 


i i 


• 

CO 






0<f4 


• 




Ou 




2 



tn 

CM •Hlr-I 

Im 

Ml 

CO 

pu 



o 

u 
cd 
u 



* * ^ 



4i 

cd 
Ou 

o 

> 

M 
O 

60 

O 
U 



M 



in 



«t 10 
•H •H 
U U 

> 



in 
to 



to 



H O CM 

<H IH O 
O MX 



e BO 

O >4 O 
w 

o o 



O P4 

M 



CO 



CO 



in 



in 




















° h 




















1 1 


• 


1 1 




• 




CO 






CO 




• 


O (H 




• 













ERLC 



lu 



in 



CO 



in 



in 











« 






• o o 


f-ijvO O : 


^ r*. • 


1 II 


H 1 


1 


1 




o m 




tt< *d ou 



in 



CO 



CI 



CO 



U4 

-a — 



17S 







in 




• 








1 


• 




CO 


•H 


• 


DU 


as 



to 



CO 



in 



■ ■ 

Kb ^ 



in 

r*- o 



I 

i 



• 

<H 

o 

4* 
U 
« 

• 



m m 

> 

o 
• m' 

CI 
O r-t 



i5 ! -H 



H 

o 

«^ 

o 
u 



U4 O 

n ac" 



f 

' s o 

M O- 



U4 

o 



• K 

n 



ERIC 



1^ 



4^ 

m 



oi 



44 
ex. 



CO 



CO 



o 




o 




• 




•o 




■ n 


• 




CO 




• 




2 



I 

•H 6 

O M 
U O 

•H 

e o 

O •H 

^ c« 
fi u 

u o 

^ c 



0 

> I 

0 l« 

• o< 
X o 
«^ u 

tf) lA O 

01 (A 
0 ^ 

C r-l 
•H O lA 

«^ c 

•H O -H 
» 4^ fi 



OI 



CO 



O 

o 



■ 



o 

■ n 

o «f 



CO 
2 



o 
•o 

■ 



CO 



in 
PI 

00 



CM 



O 



5 I 



i 



CO 
2 



in 

OI 



,%4 



to 








o 






O 










to 

• 






V 


OI 


- 












■ 


« 


■ 












Cu 


u 

















Io in 
o 



u 



IH 

-31. 



OI 

in 



U U4 



CO 
• 

2 













m 




o 


o 


0 






fi 




• 


0 




u 


bo 


a 








o o 




•o 


•H 




UJ 








0 C« 


< 


bo 


•H a 




C 


u ^ 




>H 


0 -O 


O 




i-t UJ 




c 




0 






U 


«4 




c 


c 


• u 


0 


o 




44 


u 


*l -o 


lA 




.O 0 


•H 


u 


s z 


K 


o 




ttJ 







0 
0 



fi 

6 
O 
U 

f3 
O 

o 

lU 



OI 
M 
M 



IS* tJ 



CO 



M 



in 



in 

r 



■ a a 



s 



CO 



CM 



CO 



o 

O 13 
4> O 
U«H 
0 «4 
^ 0 fi 

O d ^ 
^ lU o 

o u 
0 flu 

0 

0«H ed 

«^ » c 

•H • C 
fi 0.-H 
fi «l 0 

COM 
UX H 



•o 
in 

in 



m 
o 



a., 
flu 



CM 

in 



in 
o 



in 

CM 



to ( 



s 



U IH 



CO 
2 



0 


o e • 




u 


O 0 0 






X fi o 




0 


u 0 d 






to •H n3 




to 


O<0 




•H 


fi 


0 


10 


0 e 




10 


U 0 




0 




•H 




«o o a. 


•J 


«H 


0 




O 


S «^ M 


(M 




C X 10 


O 




•0^6 




o 


fi 0« 0 


0 


•H 




bi 






0 


0 


•HOC 


«0 


«i 


• 0 u 


P 


•H 


2 > e o< 




U 


0 o 


IH 


•H 


fl -O'H (S 


%4 




O 4i o 


0 


o 


M 15 0'H 


«4 


CO 


IM «i «4 


CO 



179 



/ 



ex. 

CO 



H 
O 



Q 



r 



CO 



0 </> 



•H ^ 
> ♦» 



♦> 4^ 

^4 in 

<H o 

Ul X 



a o 
ft 

14 o 

H O 

O O 



■lO 



O K 

M 



in 



to 



HO 



6 ^ 
^ o 



10 



10 



♦> o 

■3 O 3 /-N 
*d .O O 

O e9 O ftf • 

r-l r-« O -rl tU 
Ctf O P« 
♦1 O f4 10 C 
•H w O O (tt 
P« ^ 'H 

O a C f4 'H 

X (t 'H O tf) 

U U 

O 60 C 4: 

4> o d o a« 

M P« P« ctf 



9 .«a 



to 



in 



I 



CO 



CO 



in 

CM 
I i 



00 

ko 



CO 



Ul 



a 



180 



\ 



bo C 



I 



4ii ^ 



Examination of Table 44 indicates that the following 
unique combinations of ABSENCE of hospital characteristic 
and RECEPTION of consultation (interaction) appear to 
produce a substantially higher number of hospital 
education changes : 

Hospital Administrator with Degree in Hospital Ad* 

(Fi = 36.37, p<.001) 
Functioning Trustee -Me a ical Staff -Administration 

Committee (Fi= i . 24, p^. OS) 

Another combination whi ch appears to produce a higher 
number of hospital education changes is the ABSENCE of 
consultation and the PRESENCE of solicitation of 
assistance from New England medical schools in develop- 
ment or implementation of physician education programs. 
(Fi=S.S3, p. OS). 

The following combinations of both PRESENCE of the 
hospital characteristic and RECEPTION of consultation 
appear to produce a substantially higher number of hospital 
education changes : 

Post-operative Recovery Room (Fi = 11.51, p<.01) 
•Internal Medical Audit (Fi = 4.87, pCOS) 
Use of Extramural Nursing Consultant tFi=4.4l, p<.05) 

The prime interpretation of data of this sort suggests 
that this unique combination of attribute of the hospital 
and the reception of educational consultation activity 
produces a type of synergistic effect. The underlined 
cause of such synergism might very well be that edu- 
cational consultants respond to the unique hospital 
attribute in question and make recommendations and/or 
stir up the hospital staff so that educational changes 
are pursued more intensively by the hospital. 

Further exam inat ion of Table 44 indicates that 
whether experimental or control hospitals did or did 
not have the following facilities, a significantly greater 
number of hospital education changes occurred in EXPERI- 
MENTA.L hospitals (effect of treatment): 

Intensive Care Unit (Fc=4.17, p<.05) 
Outpatient Department (Fc*5.08, p<.05 
Electroencephalogram (Fc=6.74, p<. 05) 
Home Care Program (Fc=16.05, p<.001) 
Dental Facilities (Fc=10.92, p<.01) 
Pharmacy (with pharmacis t) (Fc = 4.11, p<,05) 
Premature Nursery (Fc = 5.75, p<.05) 
Psychiatric Unit (Fc=4.71, p<.05) 

An Approved Medical School Affiliation (Fc=4.32, p<.05) 
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Extramural consul t ants : 

Physician (Fc«6.20, p<.05) 

Allied Health (Fcs6.47, p-^05) 

Hospital Administration (Fc=5.42, p^.OS) 

Interhospi tal education program 5 for ; 
Physicians ^(Fc5=S.05, p^.OS) 
Nursing (Fc = 5.93, p<^.OS) 

Findings such as these are associated with a greater 
number of educational changes for several possible reasons. 
For one, the attribute in question may intrinsically be 
complex enough for undergoing such change as to require 
special educational activity for its successful contribu- 
tion to patient care. In addition, it may be that such 
activities or features of hospitals intrigue the attention 
of the medical consultants and are the object of dis- 
cussion and/or recommendation which ultimately lead to 
educational changes . 

An alternative explanation might be that th^se features 
of hospitals have particular relevance to patient care 
activities and are more likely to be associated with 
educational change rather than patient care-oriented 
activities. Suffice it to say that data such as these 
might very well be considered as a- series of suggested 
hospital attributes that could be utilized as criteria 
for selection of hospitals to receive educational consul- 
tation with the tactical objective that such hospitals 
are more likely to be instances of "success" for the 
educational consul tat ion . In somewhat simi lar fashion 
these findings might lead to the development of a 
checklist of priority issues to be discussed at a given 
hospital by an educational consultant in the course of 
continuing medical and allied health continuing education. 

Examination of the data in Table 44 relating to the 
effect of the independent variable, i.e., hospital 
characteristic, indicates that regardless of the reception 
of educational consultation a significantly greater number 
of educational changes were found in those hospitals 
having intensive care units, coronary care units, EEG 
units , home care programs , inhalation therapy depart- 
ments, dental facilities, rehabilitation , programs for in- 
patients, the use of external consultants for allied 
health programs and the use of external consultants for 
the continued use of educational programs , hospital 
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salaried directors of medical education, presence of 

education committees , presence of commi ttees or 

directors of hospital -wide education and training, 

solicitation of assistance from New England medical 

schools for the development of physician education programs, 

presence of residency and/ or internship programs , 

and participation in inter-hospital education programs for 

nursing • 

With regard to Table 45, the data indicates that, 
ignoring whether hospitals received or did not receive 
consultation, a significant number , of hospital education 
changes occurred in those hospitals having Full Time (as 
compared to Part Time) chiefs in the following areas: 

Radiology (Fr=18.46, p<..001) 
Pathology (Fr = l8.81, p<..001) 
Medicine (Fr=6.64, p<.05) 
Pharmacy (Fr=5.13, p<.05) 

Ignoring whether hospitals received or did not receive 
consultation, a significant number of hospital education 
changes occurred in those hospitals having (as compared 
to not having) a Chief of Surgery (Fr=5.80, p^.05). 

This roster of hospital § medical staff attributes 
associated with greater frequency of educational changes 
is not particularly surprising in that the roster of 
attributes contains either examples of such educational 
changes and/or the institution's vehicles or resources for 
implementing such educational changes . In short , those 
that have the facility for the introduction of educational 
changes are likely to manifest such changes. 

Another way to look upon the data in Tables 44 § 45 
is to note the instances (rows) in which the effect of 
treatment or the presence of educational consultation is 
not associated with a significant influence on the number 
or percentage of educational changes reported by the 
institution. Such instances might be perceived as 
"failures" or "instances of lack of success" on the part 
of the educational consultant in stimulating or 
expl ici tly recommending continuing educational programs 
for the hospital staff, and as such might very well 
represent a dimension of training and/or consultation 
strategy to be stressed during the course of educational 
consultation programs . The comments to be made in 
regard to the data in these tables re late to the pattern 
of somewhat discrepant findings when the dependent 
measure is either number and/or percentage of educational 
changes recorded by the participating hospitals. 
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CONCLUSION S SUMMARY 



Results of the three year study of the impact of education 
consultation on development of hospital -based CME programs 
indicate that, when the impact of consultation was evaluated 
under controlled conditions , critera of effective consul- 
tation were met: 1) 18 months after consultants had 
submitted their recommendations experimental hospitals had 
fully implemented 60 percent of them; and 2) a statistically 
significant greater number of physician education changes 
occurred at hospitals receiving consultation than at a matched 
group of hospitals not receiving consultation'. 

The later criterium is further reinforced by the find- 
ing that a higher degree of exposure to PHI consultation 
results in a greater change rate, as evidenced by the 
statistically significantly greater number of physician 
education changes occurring at those experimental hospitals 
receiving three or more visits by medical education consul- 
tants when compared with those hospitals receiving less 
than three visits. 

Additional support for this finding regarding 
influence of exposure to PMI, again appears upon inspection 
of recommendation implementation data. Consulted hospitals 
receiving three or more visits by consulting medical 
educators implemented in full or part a significantly 
greater percentage of the recommendations transmitted to 
then . 

These outcomes would seem to support the conclusion 
that limited inputs by physicians acting as education 
consultants can signi f icant ly facilitate development of 
hospital-based continuing medical education programs . In 
view of the fact that experimental hospitals, widely 
distributed over four New England states and ranging from 
less than 25 beds and 10 physicians to over 300 beds and 
ISO physicians, were able to demonstrate significant 
increases in CME program development, one may reasonably 
infer that with education consultation, community hospitals 
regardless of size or location can develop and maintain 
CME programs. These conclusions have direct implications 
for professional hospital medical educators who a.; e 
responsible for developing programs at their own hospitals 
and who can provide the consultative assistance to 
facilitate program development at other hospitals that do 
not yet enjoy the resources' of a professional hospital 
medical educator . 



♦Percentage refers to number implemented out of number 
transmitted . 



While study results showed that relatively small 
consultative inputs could stimulate program development, 
and although we suggest that the role of education con- 
sultant could be assumed by established medical educators, 
it still may not be feasible to supply such help to all 
hospitals in all paits of the country. Furthermore, 
in the early phases of the study, both consultants and 
hospital CME program coordinators repeatedly expressed 
a desire for more readily available information of the 
what-to-do and how-to-do-it variety. They wanted a 
comprehensive , yet flexible framework upon whi oh they 
could base their approach to the problem of developing 
relevant education programs. Their desire was for 
guidelines and practical suggestions which they could relate, 
adapt and apply to the programs for which they were respon- 
sible. 

We integrated insights gained in the present study 
with experience and data accumulated by PMI since 1961 
while collaborating with over 100 community hospitals 
in attempts to build better education programs. The 
outcome was the creation of a book entitled Continuing 
Medical Education at Community Hospitals : A Manual for 
Program Development ."' The manual provides guidelines 
for: 1) developing community hospital -based programs 
of continuing medical education; and 2) delivering 
education consultation to community hospitals* Presented 
in the manual is a Systematic Approach to Developing 
Education Programs , which is a problem-solving application 
of basic scientific methodology. Development of 
relevant hospital -based continuing education programs is 
viewed as the problem and is approached through a sequence 
of eight steps : 

1. Marshall commitment, support and personnel 

2. State overall goals 

3 . I dent if y educat ion needs 

4. Formulate' objectives addressed to identified needs 

5. Consider potential elements of an education program 

6. Determine local resources and needed supplemental 

resources 

7. Structure and implement program based on objectives 

8. Evaluate program relative to objectives and goals 



^Written by Norman S. Stearns, MD, Marjorie E. Getchell, 
MA, and Robert A. Gold, EdM; and published by the 
Massachusetts Medical Society as a supplement to the New 
New England Journal of Medicine , Volume 284, Number 20, 
May 20, 1971. A cop/ of the manual accompanies the present 
report . 



The manual also includes a chapter on principles, . 
techniques and procedures for delivering consultation and 
a section in which invited contributors, including Drs-. 
Angeio Angelides, Clement Brown, Daniel Fleisher, Jack 
Perlman, Virgil Siee and John Williamson present their 
approaches to relating quality care assessment to 
continuing medical education. 

t 

PMI*s study of continuing medical education activities 
at more than 80 community hospitals in New England 
indicates that hospitals recognize the need for CME programs 
xEnd are ready to accept responsibility for developing them. 
Our observations also suggest that there is room for 
improvement in planning and implementation of all aspects 
of hospital education programs. Desired changes can . 
be stimulated by education consultation to hospitals and 
by training of hospital •s physician education program 
coordinators . 

Outcomes of PMI's consultation efforts have been re- 
ported in terms of implementation of consultant recommenda- 
tions and in terms of reported changes in educat ional act ivi 
ties in hospitals. We recognize behavioral change leading 
to improved patient care as the ultimate demonstration 
of educational accomplishment. But as a beginning sitp» 
we will accept change that demonstrates a willingness of 
physicians to help create educational opportunities at 
their own hospitals and to assume responsibility for 
carrying out these efforts on a continuing basis. ^ 
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CONTRACT DISSEMINATION 



Following is a listing of formal presentations and 
publications in which the contract, its design, procedures, 
or outcomes, is centreJ. to the discussion. Also listed are 
published materials of PMI or other organizations and 
individuals which cite the contract in reference to their 
own work being presented. Since the publication of 
Continuing Medical Education in Coawunity Hospitals ; A 
Manual for Program Developaent , reviews and reports of the 
manual have appeared in several publications. A listing 
of these sources appears in a subsequent section. 



Formal Presentations 

"Coj:nitment and Involvement Requirements for Education 
Program Development in Community Hospitals", Norman S. 
Stearns, presented at Conference on Medical Care and 
Education in the Community Hospital, sponsored by the 
University of Virginia Medical School, September 19-20, 
1969, Charlottesville, Virginia. 

"Program Motivation and Stimulation - The Activities 
of the Postgraduate Medical Institute", Norman S. Stearns, 
presented at Conference on Goals and Techniques of Con- 
tinuing Education, sponsored by the Association for Hospital 
Medical Education, February 4-S, 1970, Chicago, Illinois. 

•'The Postgraduate Medical Institute in Continuing Medical 
Education", Norman S. Stearns, presented at Conference on 
Continuing Medical Education, sponsored by American Medi- 
cal Association, October 13-15, 1970, Chicago, Illinois. 

"The Impact of Educational Consultation on Development 
of Continuing Medical Education Programs in Community 
Hospitals", Norman S. Stearns, Marjorie E. Getchell, 
Robert A. Gold, and Martha Boucouvalas, presented at 
the 67th Annual Congress on Medical Education, sponsored 
by the Association for Hospital Medical Education, Febru- 
ary 12-13, 1971, Chicago, Illinois. (See Appendix M 
for a copy of the presentation.) 



Published Materials 



MD Training Project Planned, AMA News , Vol. 10, 
July 17, 1967, p. 6. 

Program for Continuing Education of Physicians, 

This Week in Public Health . 16 (29) 282, July 17, 1967. 

Continuing Education in ComRunity Hospitals, JAMA, 
201, (4), July 24, 1967. 

PHS Grant to PMI , Massachusetts Physician , 834, 
October, 1967. 

Stearns, NS: The Training and Use of Medical 
Academicians as Educational Consultants to 
Community Hospitals » Clinical Research . 18:481, 
ApriU 1970. 

Stearns, NS, Getchell, ME, Gold,RA: Continuing 
Medical Education in Community Hospitals - A 
Manual for Program Development. Boston, 
Massachusetts Medical Society as a supplement to 
the New England Journal of Medicine, 284 (20), 
May 20, 1971. 



Citations 



A Core Medical Medical Library for Practitioners 
in Community Hospitals, Norman S. Stearns, Wendy 
W. Ratcliff, NEJM , 280:474-480, February 27, 1969. 

NERMLS: The First Year, Charles C. Colby, Harold 
Bloomquist, T. Mark Hodges, Bulletin of Medical 
Library Association , ST (4), October, 1969, 
p. 329-337. 

Medical Ecumenicism in Portland. William L. MacVanc, 
Jr., M.D., Journal of Maine Medical Association . 
59 (S), 85-86. 

Postgraduate Medical Institute: Its Challenging 

Role in the Continuing Education of Physicians 

and Other Health Care Personnel, Robert P. McCombs, M.D. 

Massachusetts Physician, June, 1970, 34. 
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Published Materials Relating to 'Hlontinuing Medical Education 
in Community Hospitals - Manual for Program Development 



Summary of Minutes of Executive Committee Meeting, 
October 28, 1970, Los Angeles, California. 
AHME Journal , 4(1) :62, January - February 1971. 

Continuing Education: A Standoff Between You 
and the Educator? Patient Care , V(10): 31-32, 
May 30, 1971. 

Hall, JH: President's Message - A Challenging 
Chance for Change. AHME Journal , 4(3) :1, May - 
June 1971. 

Manual on Continuing Medical Education Program 
Development. AHME Journal , 4(B) :41, May - 
June 1971. 

Manual for CME Program Development in Community 
Hospitals . AMA Continuing Medical Education 
Newsletter , III (2):2, August 1971. 

Community Hospitals Urged to Meet Continuing 
Medical Education Needs. Cross-Reference , 
1(9) :10, September 1971. 
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CONTRACT "SPIN-OFF PROJECTS 



CONTRACT NO. NIH 70-4149 (PH 108-69-47) 

Develop and Evaluate the Consultation Method 
in Establishing and Maintaining Educational 
Programs for Physicians in Those Hospitals 
Located in Depressed Areas • 



A direct outgrowth of the present contract was 
contract NIH7C-4149 (PH 108-69-47), in which PMI developed 
and used educational consultation an attempt to stimulate 
relevant programs of physician continuing education at 
three hospitals located in depressed areas. Consultation 
procedures used in the present study (NIH70-41S0) to 
promote conventional physician education programs at 
community hospitals, were modified and utilized as a 
teaching technique and as a device to engage medical staffs 
in activities relevant to health care problems of the poor. 

An interdisciplinary consultant team of physicians 
and behavioral scientists, expert in medical education, 
community medicine, health care delivery, and evaluation, 
met in consultation-teaching seminars with a physician- 
led team of hospital personnel, composed of physicians, 
nurses, allied health professionals, administrators and trustees 
from a 600-plus bed community hospital located in an urban 
ghetto. The two teams examined the poverty area community 
and the hospital's relationship to it as a ''patient" with 
the health care; problems of such a "patient" the subject of 
discussion. Significant in-hospital communication gaps 
were uncovered and addressed. 

In addition to their role in planning formal education 
activities, the consultation sessions served directly 
as educational forums themselves. Subsequent changes in 
hospital policy and establishment of additional relevant 
education and service programs point to potential utility 
of the consultation method. 

Members or the consultant team also met with physicians 
and administrators of two smaller community hospitals, one 
located in a low-income, high population density, white 
community and the other in an economically declining, former 
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"mill town" possessing a significant Spanish-speaking 
population. In the former case, the consultation achieved 
its program development objectives only after considerab le 
expenditure of efforts to gain the hospital's trust and 
acceptance of the need for such continuing education. In 
the latter case, despite repeated attempts, the establish- 
ment of a lasting consultative relationship was not achieved. 

As reported to the 1970 Medical Services Conference of 
the Council on Medical Service of the American Medical 
Education in Boston, November 28, 1970*, this pilot attempt 
to stimulate the development of continuing education 
programs for physicians at hospitals in depressed areas 
through consultation has indicated the potential feasibility 
and utility of the approach* Despite difficulties encountered, 
experience points to the need for its use to bridge 
knowledge , service and inf ormat i on gaps between innovative 
activists, already deeply involved in the health care 
problems of the poor and in altering existing health care 
delivery systems, and the majority of hospital medical staffs 
which are as yet uninvolved. 



LIBRARY DEVELOPMENT PROGRAM 



In response to needs expressed by hospitals participating 
in the present study** for help in developing medical library 
facilities, PMI undertook a library development program in 
cooperation with and supported in part by the New England 
Regional Medical Library Service (NERMLS) at the Francis 
A* Countway Library of Medicine. The first phase of this 
program resulted in the establishment of a list of 47 texts 
and 39 journals which can be considered to be the basis 
for a Core Library for use by practitioners in community 
hospitals* This list is based upon recommendations 
elicited from more than 400 specialists from regions 
throughout the country represent ing 41 areas of medLcal 
practice. 



*A refined version'of the paper is published in 
Medical Care , 10(1), January - February, 1972, under 
the title "Education Consultation: A Team Approach To 
Stimulating Hospital Staff Involvement in Health Care 
Medical Problems of the Poor*" 

♦♦Contract No. NIH 70-41S0 
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Conceptually, basic requirements of the core library 
are that all major areas of medical practice be covered, 
that a minimal number of high-quality journals appropriate 
to practice be included and that all materials be readily 
accessible and logically arranged for use by physicians. 

Acceptance of the concept of the medical core and 
suggestions that similar materials should be available 
for nurses and other allied health personnel, stimulated 
development of an Integrated Health Science Core Library** for 
physicians, nurses and allied health practitioners. The 
Integrated Health Science Core Library collection includes 
87 books and 74 journals recommended by specialists as 
appropriate for practitioners at community hospitals. 
The collection continues to support the concepts, first 
suggested by the Medical Core, that materials available 
in a hospital library should be authoritative, current 
and comprehensive, covering all major areas of practice. 
' It suggests that hospital libraries should be used by the 
entire health care team . Arrangements of materials within 
the total collection has been purposely structured to 
facilitate and en.courage their interdisciplinary use. 

In addition to compiling the Core collections, PMI, 
in conjunction with the New England Regional Medical 
Library Service (NERMLS) has also worked extensively with 
community hospitals to train relevant persojinel; to 
stimulate development of libraries incorporating Core 
Library principles; and to evaluate changes in the provision 
and use of library-information facilities and services 
at hospitals participating in the Library Development Program. 



Stearns, NS, Ratcliff; WW: A Core Medical Library 
for Practitioners in Community Hospitals. NEJM, 280:474- 
480, February 1969. 



**Stearns, NS, Ratcliff, WW: An Integrated Health 
Science Core Library for Physicians, Nurses and Allied 
Health Practitioners in Community Hospitals. NEJM , 
283:1489, 1970. 



APPENDIX A 



NORMAN S. STEARNS, "POSITIVE APPROACHES TO 
CONTINUING MEDICAL EDUCATION IN COMMUNITY 
HOSPITALS", New England Journal of Medicine , 
277:1341-1344, 1967 



SPECIAL ARTICLE 



POSITIVE APPROACHES TO CONTINUING MEDICAL EDUCATION IN COMMUNITY 

HOSPITALS* 

NOKMAN S SifXRNS. \| l).t 
KOSiON 



THE P()st«ra<liMte Medical IiistitiiUs estahlislied 
111 1953 under the sponsorship of the Massa- 
chusetts \1e(hc«il Societ>. was designed to enconniKe 
and, at least in part, to provide for the continuing 
e(htcatioii ol ph>sicians.' The Institute heheves that 
coiuiiinnity hospit.ils will he the pinnaiy f'ocns of 
coiitiniiiiiK education activit> for pli> sicians in the 
future. It IS expected that practitioners will coiitiuiie 
to educate themselves !>> reading and 1)\ attendance 
at society uieetin^s and formal c(Miises. Such indi- 
vidual efforts, however, will truly he recognized and 
rewarded in the hospital setting .iiul particularly in 
hospitals with uieaniiiKful education programs of 
their own. 

The present Koal of the Institute is not .simply to 
trausniit a few facts to a limited iiuiiil>er of physi- 
cians thron^li a series of isolated lectures. j»raiid 
rounds or television presentations, hut r.itlier to 
cultivate an atmosphere of euliKliteiied interest in 
^ood medical piMctice through estaldishnient of 
co-oidinated hospilal-hased programs, with the use 
of all anpropriate avadahle educational technics, 
which will coutiiiiially challenge and enhance the 
capahilities <if every staff iiiemher. The pro^iani 
slioiild serve as a means of coiiiiiiuiiicatioii for all 
staff iiieiiihers. inclndiuK nurses and other para- 
medical personnel, and slionld stress and stimulate 
total patient caie .is a co-operative endeavor. The 
programs that have the hest chance for continuing 
success are those* that prov ide for responsihle leader- 
sliip at the local level and make full use of pro- 
fessional and teaching talents of existing staff 
iiKMiihers. In addition, ^nest cdiisidtaiits should al- 
ways he used on a reunlai hasis to siippleuieut local 
strengths. From a coiiuiiiniity standpoint, the piac- 
tical plaiiiiiiiK appnnicli uill leco^ni/e the <ireas of 
practice that .ne ade(iiiateK covered as well as 
those that are not. Moreovei. those interested and 
capahle of heconiiiiK proficient in specialty aieas 
should he eiK-uiiM^ed to hecoiiie more adept so that 
tlie\ iiKi\ fulfill coiiiniuiiity needs as teachers as 

•Prescnici! on M,»> 17. 1967 .ti the dnnuul iDeeiing of ihc M.»s«wJLhu- 
setts Mcdic:il Society (reqiiesls for reprints should he .iddresscd to the 
Postpri»du:»tc Medtc:jl Institute. 22 hcnw.iy. Boston. Mdss.ichu setts 
022 l.M 

tMedic.il director. Postgntduaie Medtc.tl Institute, clinicil associate in 
medicine. Harvard Medical $ch(H>l.' assisting physician. Second .ind 
hourth < Harvard) Medical Services. Boston ( ily Hospital, director of 
medical educalion. Newton- WcHesley. Waltham and I,, f- Quigley 
Memorial hitspitals. acling director. Pri-State Regional Medical 
Program (M.iss.ichusclts. New Hampshire and KhiuJe Island) foi Heart 
Oisease. ( ancer .ind Stroke 



well as piactitioiiers. If this opportnmt\ for fnitliei 
training cannot he provided at the local level it 
should he sought at academic centers tliroimli co- 
operative aiTiiu^emeuts. 

To achieve tliese Koal.s on a hroad scale., the 
Postgraduate .Medical Institute as ii widel) supported 
central agency provides hotli stinmlns and necvled 
resources enahlin^ iudi\ idiial coiiiiiiiiiiit\ hos- 
pitals, rej^ardless of size, to improve existing edu- 
cational programs or to undertake new ones in a 
manner that iiiiKht otherwise he iinpossihle because 
of limited availability of experienced medical edu- 
cators or of competent teachers and consultants. 
Thus, the Institute is interested in providing con- 
sultation services for development of individual 
hospital programs, as well as coiitiiiiiiii« such exei- 
cises as ^rand-round coiisnltaiit programs, lecture 
courses, television productions (Boston Medical 
Reports'*) and televisioii-leleplione two-way confer- 
ences. 

Practical Program Planning 

The experience of the Institute indicates that au\ 
hospital desirous of iiitrodiicin^ new programs of 
coiitimiiiiK medical education, or of improving ex- 
isting ones, can take specific steps. 

First of all, the people respousil)le for iiiitiatui^ 
e(hicatioiial programs should leview the pertinent 
literature.'*'* References cited pro\'ide (j^eneral and 
specific) hackKroiiiid material that will he helpful to 
those concerned with setting the course of iiiiy new 
iiudertakiii^. 

The hospital, includiiiK its medical and adminis- 
trative staffs and trustees, should define overall pro- 
gram objectives. A freewheeling and somewhat 
idealistic approach is euconraKed at the outset. A 
written list of all iniiiiediate and loiiK-ran^e objec- 
tives slioiiKl be L*onipiled 

llie liospit«il should .issi^ii. elect or hire a physi- 
cian to assume responsibility for overall program 
development at the local level. Whenevei feasible, 
tins person should be salaiied. In *ui>' event., he 
must be assiiied oi strong moral support on a ton 
tiuiiin^ btisis by a niajorit> of the staff, the adminis- 
tration and the trustees. 

A complete outline of all existing hospital <iikI 
couiiiinuity -service activities re(|iiiriii^ participation 
b> staff physicians is essential. The P(ist^r<idnate 
.Medical Institute provides forms to expedite iiifor- 
ination-Katlieiiii^ efforts. Information obtained will 
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cnahli' .1 realistic .ippi.usal ol what can he expected 
III the Nva\ ol «i('tnal participation in existing oi pio- 
posed activities and will ideiitily existing and axoid 
fntnie eonllicts ol time and place in the program 

It is esseiituil to learn as nnicli as possihie ahont 
each staff nuMnhei. particular!) ainnit his willingness 
to participate iii an> new piograin UsiialK. a de- 
\oted lew have alone home the hiiiden ol existing 
or pievionsix- attempted programs, with iiiinmial 
support and little th.niks Iroin the staff as a whole. 
This situation nnist be recognized. Any new plan 
innst emphasize the absolute necessity ol creating a 
program based on shared responsibility lor its opeia- 
tion Koi these le.isons. a staff <inestioiiiiaire is 
provided by the Institute, and hospitals aie le- 
<|uested to send them to all staff nieinbers together 
w ith a letter inloriuiiig each physician that his sup- 
port is being sought as an essential participant in an 
overall hospital activity. In addition to biographic 
d«;ta .uul cnrricninm v;tae. the <iuestioiiiiaire re- 
(piests specific answers to the following <piestioiis: 

All' Mm iniertMC(i in piniici|Miiii^ iii le.uliiiig .uiixities 

ai /^'^ hospii.il (i e. toniids. ( iiiiiis. tiHinrcs, cu .)} 

\\oiil(i \c>u Ik* willing lo .isstitne tes|HMistl)ilii\ toi tiM(li- 

II ii; sessions ,is "iiisti lu lot"? II \es. pliMsi- indicate \om 

paiiKutai niteiest. 
Would Mui \yv Nsillin^ lo aiiend s( lu-diiled wm Inng 

a< mines (oitduded I>n oihcis at tins liospital iis a pan (d 

\tm\ own (onniitiini; ediu.iiiiMi pto^iaHi!" II ws. indKaie 

nunilH'i (»l lionis |k'I week im tiionili 
Please sni;i;esi daNs and inncs \fMi pielei tm sthednled 

ediMational tni-etini;s 

UiMdd Non Ik* willing to .illow pteseiilaiion and discns- 

siiMi td \"in pti\aie paiienis. n}>on leipiesi. toi teadiing 

pin poses ai iintiids and (oideieiKCsi' 

('onstrtieti(m ol" the new program begins with a 
review of all available iitloriiiation. Analysis of 
larger hospitals with multiple activities ean be 
made initially on a departmental basis. Smaller in- 
stittitioiis with lew activities can be analyzed on a 
total hospital basis Iront the outset. Idealized objec- 
tives are boiltnl down to ones that seem practical 
and immediately attainable. Existing activities that 
wanaiit contiiitiance, with or without modification, 
are recorded on a simple master-plan schedule lot in 
provided by the Postgraduate Medical Institute. 
Activities not considered worthwhile are abaiuhmed 
Suggested new activities are discussed, and those 
that seem leascmable ol accomplishment are 
recoided tm a new master-plan loriit. Conflicts of 
time, place and |)ersoit beeoiiie iiiiiiiediately appai- 
eiit. When scliedtil 's are made on a specialty serv- 
ice basts, care iimsl be taken to integrate iuter- 
sei vice activities whenever possible and to ideiitily and 
co-oidinate lull stalf activities on a se|)aiate master- 
plan loiiii. 

Opportunities lor establishing uteaiiiiigliil affilia- 
ti«)ns on a coiitiimnig basis between cominunity 
hospitals and iniiversity centers lor purposes of 
edncatioit. coiistiltation and. in s<mie institutions, 
p.itieiit care sitotihl be explored.** Integrated fellow- 
ship, residency and practitioner training progiaiiis 



should be consideK (l.** The poteiiti.il lole oi regioii- 
.tl medical piogi.mis in eflec tinu affiliations foi ap- 
propri.tte ptiiposes should be discussed locall) .md 
with represeiitatiNcs of die Regumal Medical Pro- 
gram Staff."''" 

All ineiii!)ers of the stalf on each senu e should be 
strongly urged to p.trtuipate in teaching .tctiMties 
by virtue ol assigned respoiisihility foi spciifit- 
rotiitds. eonleiences .tnd clinics Ktill stalf p.trticipa- 
tion will ensm'(* an ecpntable distlibtition of .tssiun- 
iiieiits. thereby preventing excessive d(Muaiids on 
the tune of any lew members. Individual piactitioii- 
ers slumld be eiic<niraged to associate themselves 
lor several mouths each year with a specialty gn)tip 
organized for patient caie and teaching purposes oi 
with gtiest-coiistiltaiit lotinds held regularly in spe- 
cialty fields. These small groups slumld provide a 
nucleus of physicians who will find appropriate case 
material and be present when cases are discussed. 
Unless bouse officers are available, these practition- 
ers slumld present selected cases to visiting con- 
sultants or to <iualified staff members assigned to 
conference or rotiiid responsibility in advance. Reg- 
ularly scheduled bedside rotiiuls or at the very least 
patient oriented grand nmiids slumld he encouraged 
in all hospitals and at every op|M)rttiiiity. 

The ciirreiit ami projected lole ol service chiefs 
should be ccmsidered in relation to formal teaching 
activities and also to responsible leadership in stiiii- 
ulatiiig heightened interest in <|tiality care on a 
continuinil basis When stieli a step is deemed ap- 
propriate for discussion, the potential value of sala- 
rie<l service chiefs on a part-time or geographic 
fnlltiiiie basis slumld be given serious coiisideratioii. 

If resistance to the use of private patients for 
teaching purposes exists (for example, presentation 
at grand rounds), efforts sh<mld be made at the local 
level to dispel any such fears. If these efforts by 
local loaders are not successful, help froiu respected 
educators outside the couiiuuiiity should be sought. 

Most community hospitals have a eonstiltaiit staff, 
aiul Kjcal practitioners regularly use- certain other 
distinguished physicians for referral purposes. .All 
coiistiltaiits who directly benefit from tlieii associa- 
tion with c<niiiiiiiiiity-liospital staff physicians slioiild 
be re<|tieste<l to contribute to the overall program by 
assnniing limited responsibility for designated 
teaching sessions. 

Participation by staff radiologists and pathologists 
enhances any graiid-rotiiid program. These key peo- 
ple should he given advance notification <d cases to 
l)e presented in time to ensure propei piep.tratioii 
of material. 

The Postgraduate Medical Institute notifies local 
cliairiiieii of the baekgioinid, interest, academie af- 
filiati(ms and titles <d* their speakers by postcards 
sent ir. advance of iiteetiiigs. Hecpiireiiieiits for sliile 
and f^«)vie projectors, blackboards or other teaching 
aids are requested ol* each speaker, and the local 
chairmen are so informed. Arrangements for these 



facilities should be made in advance of the meeting 
by the local chairman. The Institute also notifies 
speakers of the nature of teaching sessions, travel 
directions, number of physicians expected to »itten(l 
(when known) and the time, date and duration of 
each session. 

All programs should include some form of 
mortality conference and audit of service as recjuired 
for hospital accreditation. The (juality of medical 
care can also be assessed in an on-going program of 
current case medical audit." Information obtained 
from usually routine-staff committee meetings (in- 
cluding utilization review, tissue, infection cwitrol 
and tumor cHimmittees), if properly prepared can 
provide the liasis for periodic full-staff audit meet- 
ings. Self-evaluation and self-criticism should l)e 
fostered as a privilege of physicians in practice. The 
purpose of current case audit should be education 
rather than control, and improvement of the quality 
of care should be the essential objective of audit. 

Two simple technics that have promoted interest 
are the "X-Ray of the Week," displayed in the x-ray 
department, and the gross or microscopical "Speci- 
men of the Week," in the pathology department, 
with explanatory notes or with an invitation to 
"make a diagnosis.*' Answers are provided in writ- 
ing or directly by the radiologist and pathologist. 

Teaching aids such as motion pictures, pro- 
granuned instruction devices and talking slides 
should all \ye considered and used to fill specific 
needs. 

Hospitals in many areas may round out or supple- 
ment their own programs by direct participation in 
television, telephone and radio-network programs in 
a regular fashion. 

Medical-library facilities and services should be 
reviewed, evaluated and updated. Specific consulta- 
tion advice should be obtained for lil)rary develop- 
ment when facilities and services appear inade- 
(]uate. 

The number of hours per week or per month of 
scheduled educational activities should be carefully 
reviewed as the fonnal program takes shape. It is 
best to begin with a reasonable program in terms of 
time recjuired, with staff questionnaire responses as 
a guide to what can be expected in the way of physi- 
cian participation. 

Nurses, technicians and other ancillary personnel 
should be invitet! to as many medical staff teaching 
activities as possible even though the educational 
content of sessions is geared to physicians. 

Staff physicians should be involved on a shared- 
responsibility basis in educational programs of 
nurses, technicians and other ancillary |>ersonnel. 

With the aforementioned considerations clearly in 
focns, the working program is constructed. After all 
elements of the program have been outlined and 
incor|)orated on a master-planning sheet, the formal 
educational sessions should be recorded on a calen- 
dai foun. The Institute encourages plannint; on a 



basis of six to twelve months whenever possible. 
The calendar form, with holidays .md local "no- 
ineetiiig ' days clearly noted, is the final pKuuung 
instrument The program u\ action wdl reflect this 
planning. 

Those planning the programs need not be dis- 
couraged by **jK)or attendance*' at any few sched- 
uled sessions, but should pay attention to overall 
participation in the continuing pmgrani New uaviis 
to involve nonparticipants, and to encourage and 
reward continued participation by those activeK 
mvolved in the program, should be constantl) 
sought. 

Summary and Conclusions 

Every community hospital should have a sound, 
broad and well phmned educational prograin pri- 
marily to provide for continuing staff education as a 
means of encouraging better patient care The Post- 
graduate Medical Institute sponsored by the Massa- 
chusetts Medical Swiety has, since 1953, encour- 
aged and in part provided for the continuing 
education of large number of practitioners through 
a variety of technics. 

On the basis of the Institute's experience, pro- 
gram planners in community hospitals should take 
the following steps to develop new educational 
programs: review pertinent literature;, discuss pro- 
gram objectives; mobilize full sup|>ort for a local 
program director; critically analyze ongoing educa- 
tional activities; appraise honestly the contribution 
of each staff meml>er to existing and to potential 
new program activities as teachers and participants: 
plan a new program based upon realistic apprecia- 
tion of time commitments by all participants, con- 
sider affiliation with other institutions for nnitnal 
benefit in education and patient-care endeavors; 
c^)nsider every device to encourage full staff partic- 
ipation in various aspects of the overall program on 
a continuv.ig basis; consider current and projected 
new roles of service chiefs, encourage full utilization 
of private patients in teaching at bedside and grand 
rounds; use existing staff consultants in formal pro- 
grams and introduce guest consultants on a regular 
l)asis to supplement local strengths or to meet local 
needs; include mortality conferences and current case 
audits in the formal program; involve nurses and 
paramedical personnel whenever possible: and en- 
courage planning of and fix responsibility for all 
aspects of fonnal activities well in advancx* of 
scheduled sessions. 
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THE EDUCATOR-IN-CHIEF 

Kin'( ATio.N sans effort is a technologic iitopia for 
which many pine: under the pillow of rest, th^ 
(iormo-digest pertly murmuring and filtering infor- 
mation, like detergent innueudos, mto the perma- 
nency of the unconscious. Or the digital diviner, with 
a console of keys that need but proper pressing to 
display in appropriate sequence the differential 
diagnoses. But such devices, should they ever be- 
come available, would incidentally do away with 
the need for educated men of medicine. For his 
future preservation, as well as for his present com- 
petence, the physician must toil at self-education. 

Elsewhere, Steams outlines how it should be done 
at a community hospital. To this end, its corporate 
body of professionals, no less than the individual 
physician, must commit itself to both ergs of extra 
effort and volts of abnegation. Doctors are too busy to 
assume implementation of a hospital-wide education 
program in addition to their other duties. The im- 
plementation has to bect)me the responsibility and 
privileged province of one individual. The bound- 
aries of his province must, moreover, be precise. 
They must include the right to visit, converse wijth 
and use for teaching all patients in the hospital. 
This provision is especially crucial if the organiza* 
tion of the hospital includes house officers at any 
level. The Educator-in-Chief (equivalent to any 
physician-in-chieO should have the wisdom and 
compassion to know which patients are by nature of 
their condition not candidates for his program. 

To bolster this indispensably wide authority, and 
to prevent any possibility of his interfering with the 
basic relations of patient and private physician, the 
Educator-in-Chief must not engage in practice him- 
self. His support must he provided by an uncom- 
promised stipend, and if a community hospital 



wants an Educator-in-Chief worth his salt, the sti- 
)end should exceed the median physician income 
.if the hospital's locale. In hospitals unable to sup- 
port a full-time Educator-in-Chief, a part-time ar- 
rangement may l)e unavoidable, but it would be 
much preferable if the E-in-C spent his remaining 
time performing his special function at other hospi- 
tals, much as pathologists or radiologists often do 
now, rather than engaging in practice in his com- 
munity. For his sake, as well as that of his col- 
leagues, he should not be in competition with them. 
Only under such arrangements can he deal freely 
with the patients of others. 

Steams mentions another indispensable feature of 
the community' hospital education program. Of those 
staff physicians who participate — and the more the 
better — their commitment must be unstinted ami 
un(|ualified. If a staff member has a teaching assign- 
ment, he must carry it out, he must prepare for it, 
and he must be punctual. Otherwise the program, 
collapses. How then will he handle an emergency 
affecting one of his patients? The only solution is 
prior arrangement, as carefully scheduled in ad- 
vance as the teaching exercise itself, with a col- 
leagtie who must cover for him Dedication to and 
planning for the (|uality of the teaching progr.uu 
nuist have the same priority us time off. 

If the new regional programs .ire to pre\-ail, they 
will provide li!)rary and university services that can 
he used, as Stearns |M)ints out, to support a coininu- 
nity hospital educatioti program handsomely. Uiu- 
versit> services will never, however, be more than 
supportive unless they take over completely, and 
this is itiulesirable if a program is to enjoy the 
proud morale of self-sufficiency. The effective ne.ss of 
an educational program will he a measure of the 
home team's effort. Neither expert coaching nor 
mod<*rn e(|uipinent can succeed by it. /If. 
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•article I. Scope of Work 
PHASE I - OBJECTIVES 

a Establish criteria for selecting hospitals for the consultation 
service. Criteria shall take into account such factors as desire 
for programs of continuing medical education; distance from ntapor 
medical centers; size of the hospitals; the organizational structure 
of the hospital; the role of the hospital within the community;and 
the existing and asoired to educational facilities and resources 
for programs of continuing medical education. The criteria shall 
be reviewed and approved by the Project Officer. 

b Utilize the available data on the organizational characteristics 
aid continuing education programs of all the community hospitals 
in the four state region in order to ascertain to what extent each 
hospital meets the criteria established in Step No. 1. 

e. Using the criteria and data developed in Step No. 1 and 2. select 
a qroup of thirty hospitals for inclusion in the consultative service 
lo develop programs of continuing medical education at those hospitals. 

d With the aid of the Regional Adi/isory Boards, implement the 
consultative service to the community hospitals and institute the 
actual programs of continuing medical education at these hospitals. 
?£e contractor shall attempt to introduce the following P^^ciples 
of sound educational program development into tJie community hospitals 
programs of continuing medical education* 

(1) Secure the participation of representatives of the 
hospital's physicians in planning and carrying out the programs 
of continuing nedical education. 

(2^ Obtain data concerning the educational interests and needs 
of the hospital's physicians as a basis for planning continuing 
education programs. 

(3) Define soecific educational objectives based on analysis 
of the needs and interests of the physicians- 

(4) Select educational methods which are most likely to 
achieve these objectives. 

prior to the onset of the consultative procedures, the Contractor 
shall translate these four general objectives into specific parameters 
in order that their int-roduction into the educational programs can 
be objectively measured. 



(cope of Work 
age 2 B-2 

e. Evaluate the effectiveness of the consultation service in . 
I aiding cominunity hospitals to establish continuing education programs 
P for their physicians. Specifically, the contractor shall: 

1(1) Make a quantitative analysis of programs initiated and 
participation and response. 

I (2) Analyze the extent to which the specific parameters of 

i the principles of educational program development .enumerated 

in section 4 are incorporated into the programs of continuing 
, medical education at the community hospitals. 

' The methods and instruments to accomplish this evaluation are to 
be approved by the Project Officer. 

f. collect observations and data which shall descj£ibe some of the 
factors, both in the consultative service and in the hospitals which 
are important in either aiding or inhibiting the development of 
pJLSSs ofcontinuing medical education within the hospitals. These 
obHrvSuSL and data^hall be used to elucidate objective characteristics 
both of the hospital and the consultation service, which might 
di. ^erentially be associated with current or future development of 
programs of continuing medical education of physicians. 

q. Explore methods for evaluating the effects of both program 
lontent and techniques of program presentation on the medical practice 
of participating physicians. 

I h. Analyze, tabulate, and interpret the data. 

- PHASE II - OBJECTIVES 



i. Utilizing experiences and procedures developed in Phase I, 
continue consultation services to selected community hospitals in t^^ 
'i four state region. The consultation service shall be varied so that 

i half of the hospitals receive five visits in a 13 rapnth period 

(intensive consultation) and half only two visits in that same 13 
] month period ( minimal consultation). 

i. Continue and conclude the evaluation of the effectiveness of 
the consultation method in establishing and maintaining educational 
1 activities and the selected community hospitals. Specifically, 

such evaluation shall include: 

(1) The extent to which. the recommendations of the contractor 
are implemented at the community hospital. ^ 

(2) An assessment of the relative efficacy of. intensive vs. 
minimal consultation (as defined in 1. above) in the implementation 
of the contractor's recommendations. 

k Continue to describe the factors, both in the consultative 
sirvice and in the hospital, which materially affect extended 
maintenance of physician education programs. 
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1. Design and execute a training program for physicians who will 
provide consultation services to community hospitals in the region* 
Hie program shall be developed along sound principles of educational 
methodology* Curriculum shall include units in educational methods 
and technology, behavioral psychology, available educational resources, 
utilization of health manpower, and others* Approximately ten 
consultant trainees shall receive the equivalent of seven (7) man- 
days of training* 

m. Design and execute a training program for physicians who will 
serve as local education coordinators or directors of medical 
education* The objective of this program is to make local coordinators 
better able to cope with problems of continuing medical education 
on a sustaining basis* A local coordinator or director of medical 
education from each of the 40 participating hospitals shall take 
part in six (6) man-days of training. 
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Postgraduate Medical Institute (PMI), sponsored by the Massachusetts 
Medical Society, has for a number of years made available consultation ser- 
vices to New England hospitals, enabling them to improve existing medical 
educational programs or to undertake new ones. To date, the Institute has 
assisted over forty community hospitals in Maine, New Hampshire, Rhode 
Island, and Massachusetts. 



The goal of PMI activities in community hospitals is not simply to 
transmit a few facts through a series of lectures to a limited number of 
physicians. It is rather to cultivate an atmosphere of heightened interest 
in good medical practice through establishment of hospital based educational 
programs. To achieve this objective, PMI provides guidance to hospitals 
who wish to develop their own programs and aids in carrying out the programs, 
®-8- > '^y rnaking guest faculty consultants available to supplement teaching 
activities of local physicians as required. 



Postgra^'uate Medical Institute, through a contract with the U. S. Public 
Health Service and supported by your state medical society and the New Eng- 
land schools of medicine, is now able to offer its consultation services to a 
selected numt of community hospitals in the area without charge. The 
Institute will request each participati hospital to conduct self- surveys of* 
educational rer ^urces and needs and co meet periodically with our staff 
consultants in your hospital. Our contract requires that these efforts take 
place during the summer and fall of this year. These activities will result 
in recommendations for development or expansion of a continuing medical 
education program in your hospital. It should also be noted that this acti- 
vity is in concord with the Regional Medical Program objectives. 
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Since the number of participants is limited, it is urgent that you indi- 
cate on the enclosed form as soon as possible whether or not your hospital 
is interested and willing to receive the Institute's consultation services. 
Our deadline for selection of hospitals is June 26, 1967. 

So that you may apprise your staff of our program, I have enclosed 
a copy of this letter which we suggest you forward to the president of your 
medical staff. We will be grateful if you discuss this with your staff and 
others so that you may consider the merits and potential value of the ser- 
vice to your institution. A reprint from the New England Journal of Medi- 
cine is also enclosed as general reference material about PMI objectives 
and activities. 

Thank you for considering our proposal. We shall look forward to 
nearing from you very shortly. 



Sincerely yours. 




Norman 5. Stearns, M. D. 
Medical Director 
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Norman S. Stearns, M.D. 

Medical Director 

22 The Fenway 

Boston, Massachusetts 02215 

Dear Dr. Stearns: 

jj^g * Hospital wishes^ 



(does not wish )to invite Postgraduate Medical Institute (PMI) consultants 

for the purpose of discussing local problems of continuing medical education. 
Should PMI select our hospital for the consultation service, we suggest the 
followino dates and times for the initial visit (please list 3) _ 



We understand that PMI will contact us for purposes of arrange 
ing a mutually satisfactory date and time for the proposed meeting. Please 



^ ^ / \ . (address) 

contact (name) ^ ^— * ^ 

, f phone numb ex*) 



Sincerely yours, 
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PHASE I 

MEDICAL EDUCATION CONSULTANTS 



Daniel D. Federman, M.D. 

Assistant Professor of Medicine, Harvard Medical School. 

Stephen Goldfinger, M.D. 

Chief Resident in Medicine, Massachusetts General Hospital. 

George F. Grady, M.D. 

Senior Instructor in Medicine, Tufts University, School 
of Medicine . 

Melvin J. Krant, M.D. 

Assistant Professor of Medicine, Tufts University, School 
of Medicine . 



Morton A. Madoff, M.D. 

Assistant Professor of Medicine, Tufts University, School 
of Medicine. 



Robert P. McCombs, M.D. 

Professor of Medicine, Tufts University, School of Medicine. 

James C. Melby , M.D. 

Associate Professor of Medicine, Boston University, School 
of Medicine . 

Marvin L. Mitchell, M.D. 

Associate Professor of Medicine, Tufts University, School 
of Medicine . 



Louis N. Rashin, M.D. 

Active Staff Member, Waltham Hospital. 

George J. Robertson, M.D. 

Medical Director, Bingham Associates Fund. 

Henry E. Simmons, M.D. 

Clinical Instructor in Medicine, Tufts University, School 
of Medicine. 



Norman S. Stearns, M*D. 

Medical Director, Postgraduate Medical Institute* 



Henry S. M. Uhl, M.D. 

Professor of Medical Sciences, Brown University. 
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PHASE II 
MEDICAL EDUCATION CONSULTAxNTS 



Thomas Durant, m,d. 

Clinical Instructor, Harvard University School of Medicine, 

Daniel Federman, M.D. 

Assistant Dean for Continuing Education, Harvard Medical School, 
Assistant Professor of Medicine, Harvard University School of Medicine, 

Edward Friedman, M,D, 

Instructor in Surgery, Harvard University School of Medicine, 

Stephen Gold finger, M,D. 

Director of Community Programs, Department of Continuing Education, 
Harvard University School of Medicine, 

Norman Grace, M,D. 

Instructor in Medicine, Tufts University School of Medicine, 

George F, Grady, M,D, 

Senior Instructor in Medicine, Tufts University School of Medicine, 

Melvin J, Krant, M,D, 

Assistant Professor of Medicine, Tufts University School of Medicine* 

Vincent Lanzoni, M,D, 

Associate Professor of Pharmacology ans Assistant Professor of 
Medicine, Boston City Hospital, 

Morton A, Madoff," M,D, 

Assistant Professor of Medicine, Tufts University School of 
Medicine, 

Barry Manuel, M,D, 

Assistant Clinical Professor of Surgery, Boston University School 
of Medicine , 

Robert P, McCombs, M,D, 

Professor of Medicine, Tufts University School of Medicine, 

George J, Robertson, M,D, 

Assistant Professor of Medicine, Tufts University School of Medicine, 
Assistant Dean for Continuing Education-Tufts University School of 
Medicine, 



Cont'd PHASE II MEDICAL EDUCATION CONSULTANTS 



Clark T. Sawin, M.D. 

Assistant Professor of Medicine, Tufts University School of 
Medicine, Chief, Endocrine Section Medical Service, Boston 
V* A. Hospital. 

Henry S. Uhl, M.D. 

Professor of Medical Science at Brown University 

Associate Director of Medicine 

Division of Biological and Medical Sciences. 



APPENDIX F 



AGENDUMS AND TRAINING MATERIALS FOR CONSULTANTS' 
LOCAL COORDINATORS', AND EVALUATION INTERVIEWERS' 
ORIENTATION AND TRAINING SESSIONS 
PHASE I AND PHASE II 
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PHASE I 



POSTGRADUATE MEDICAL INSTITUTE 



A SUMMARY OF AN ORIENTATION WORKSHOP, SWAMPSCOTT, MASS. 



June 27, 28, 1967 



Attendance 



Nortnan S* Stearns, M*D* 
George J* Robertson, M*D* 
Marvin Mitchell, M.D. 
Louis N* Rashin, M*D. 
Henry £• Simmons, M*D* 
Robert McCombs, M*D* 
James Melby, M*D« 
Henry S*M* Uhl, M.D* 
Melvin J. Krant, M«D. 
Stephen Goldfinger, M.D. 



Ezra Saul, Ph.D* 
Bernard Kramer, Ph*D* 
Roger Cohen, Ph*D* 
Mr. George Nilson 

Jlr* Harold Pyke 
Miss Marjorie Kalberer 

^Mrs« Kathleen Adler 
Mr* Michael Madden 
Miss Barbara Davis 
(Recorder) 



Agenda 



Monday - June 27; 1;00 5:00 
I* Context Characterization ^ disfcussion of project setting 

1. Background of contract 

2. Relation of PMI to Regional Medical Program (RMP) 
3* Political setup 

a* Maine«*Bingham Associates Fund; RMP 
b* New Hampshire-^Dartmouth 
• c. Rhode Island-Brown Univ.; Rhode Is. Med. Society 
d. Massachusetts-Pittsf ield; Albany 

4. General schedule pattern of contract (time Chart) 

5. General outline of responsibilities of staff 

6. Questions and answers on material throughout discussion 



II. Problems 



1. Strategic issue of review^ of Bureau of the Budget-* 
memo on travel policy 

2. Plan for periodic check points, summary reporting 

3. Study plan for evaluation 

a. Discussion of dependent and independent variables 

1) Tightening of criteria for hospital selection 

2) Tightening of criteria for matching consult- 
ants with hospitals 



b. Deli elation of outcomes-planning of measurements 

c. Role of evaluator during course of consultation 
Behavioral consultants-^categories of analysis 
(for evaluation team) 



5:00>7:00 



Free time and dinner 



7; 00-9: 30 



Consultant Briefing and Indoctrination 

1. Presentation of program develo{»nent techniques- 
(outline of procedure) 

2. Review of forms 

3. Discussion of scope of work (section d) 

a. Discussion of more objective and sophisti- 
cated means of assessing need and resources^ 
e.g.# services committees. 

b. Definition of specific educational objectives 

4. Discussion of subsequent workshop with local 
coordinator 



Wednesday # June 28 

7:30-9:00 

Breakfast 
9:00-12:00 

Continuation of Consultant Briefing 
12:00-1:00 

Lunch 

1:00-5:00 

Critique of total program by 
behavioral consultants 



Suinmarv of Discussion 



The vorkshop began at 1:30 in the afternoon. Dr. Saul 
opened the session with a review of the aspirations of the 
%iorkshop which are to review the objectives of the Public 
Health Service Contract, to orient both staff and consultant 
personnel to their roles, to review and discuss a research 
design, and to make clear the consultation process. A num- 
ber of key decisions must be rrtade, for example, to arrive 
at a basic criteria for hospital selection, to attempt an 
initial assignment of consultants to various hospitals which 
inay be selected, and to discuss variables both in the consul-* 
tation process and in hospitals vAiich may affect the contract 
objectives. Finally Dr. Saul rentioned that any problems 
pertaining to the con. ct for PMI*s operation in any way, 
should be brought fort«. and discussed. 

Dr. Steams, presented the background and development 
of the contract. Dr. Stearns mentioned that the study was 
originally designed for two years. However, it was reduced 
to one year by the Public Health Service so that the results 
of the project could be made available to other institutions 
in establishing similar programs throughout the country. Dr. 
Stearns referred to a plan of progress and discussed :;he var-* 
ious phases listed, noting especially that all recommendations 
had to be completed by October 31, so that there would be a r..* 
four-^onth interval to measure the effect of che consultation 
process. 

Dr. Stearns then discussed the relationship of the Post-* 
graduate Medical Institute and this study to the Regional 
Medical Program in Maine and the Tri-*State Area. He talked 
particularly about Dartmouth in New Hampshire and the work 
of Brown University and the Rhode Island Medical Society, 
in Rhode Island. He mentioned that a portion of I-lassachusetts 
may be closely allied to Albany Medical College and its Re- 
gional Itedical Program. 

To those consultants who were new to the FMI activities. 
Dr. Stearns mentioned that PHI hopes to develop strong com^ 
munity hospital ties with the major medical centers and to 
increase the number of people who are knowledgeable in devel- 
opment of physician education, both at the local level and 
the resource level. 

Dr. Saul, asked for help in developing criteria in the 
selection of hospitajis. He mentioned that we needed a sig- 
nificant sample of hospitals in the New England area; hospi- 
tals which would be a reasonably accurate cross-section of 
hospitals throughout the country. Following a rather lengthy 
di3cussi;^n, the following three general categories were agreed 
by concensus as basic criteria for hospital selection: 
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First, coiranunity hospitals which were not involved in PMI 
educational activities; second, hospitals of varying bed. 
size so as to reflect a national distribution; third, dis- 
tance from the major med-'cal center, Boston* Dr* Saul men- 
tioned that one additic i .1 criterion which had not been dis- 
cussed earlier v;as the desire of hospitals to either improve 
or initiate proQrams of continuing medical education. In 
order to determine this, letters inviting hospitals to par- 
ticipate in the project had been sent out prior to the meet- 
ing to community hospitals in Maine, Massacl setts, and Rhode 
Island* Letters had not been sent to Kew Hampshire hospitals 
because of a request by Dartmouth that we refrain from becom- 
ing involved within their sphere of influence until after 
their own hospital project pertaining to closed circuit tel- 
evision had gotten off the ground* 

Some discussion ensued* One of the questions asked was 
about hospitals who are not accepted, and what we will do 
with them. V7ill we acknowledge their rejection or will we 
give them literature materials and no consultation, or will 
there be a control group. Dr. Saul suggested that there be 
three groups of hospitals: those who are accepted for the 
project, those v;ho are not accepted, and lion-respondents. 
Dr. Krant suggested that hospitals might be matched between 
those who have accepted and those who have not accepted 
based upon the criteria. Dr. Krant mentioned that if we 
want to find out if the consultation service has. any impact, 
we need to match certain characteristics about the hospital 
and find which variables make a difference. 

Following considerable discussion and some digression from 
the subject. Dr. Kramer, weighing the problem of consultation 
against no consultation, mentioned that it would be an ex- 
pensive procedure to answer this question, and we should pro- 
ceed upon the assumption that the Pi4I consultation is better 
than no consultation at all. He stated that we should put 
aside that question and design a comparative approach rec- 
ognizing all the different activities going on in the hospi- 
tal community from whatever source, i.e. the Regional Medical 
Program as well as PMI and whatever else might be going on 
in the hospital. Dr. Kramer sugge ad that we might compare 
the consultation procedures outli* , by Dr. Stearns with 
some modification of that consult.- t.i.on procedure; for exam- 
ple, we cculd send out literature only to a selective number 
of hospitals. From this came a discussion of a need to 
knov/ the possible outcomes of the consultation visits. 

Dr. Kramer mentioned that rot only do we need to know 
or be able to suggest possible outcomes, but we have to have 
a category of unintentional affects which may result from 
consultation visits. Because of the time limit of this pro- 
ject, of course, the extent of unintended outcomes or even 
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intended outcomes is limited to merely the Penod for as- 
sessing the impact of the service (roughly ^J?") November 1 
to February or march) and that further investigation of these 
outcomes might be a possibility for future study. ^rant 
noted the dilemma of human behavior and said that one year 
was a very limited time to come up with very much. Dr. Stearns 
suggested that the public health service also wants to find 
out how PMI gets its foot in the hospital door and also 
wants to increase manpower familiar with development of edu-. 
rational programs in community hospitals. 

At this point the meeting adjourned for dinner and re- 
convened at 8:15 p.m. 

Dr. Saul began the evening session by briefly reiterat- 
ing major points discussed during the afternoon session. He 
asked Dr. Stearns to outline the procedure for consultation. 
As Dr. Stearns discussed the consultation procedure in de- 
tail a number of problems arose through questions. First 
was the limited time, e.g. two visits, before making recom- 
mendations and that recommendation had to be completed by 
October 31, which seemed rather restrictive. The limited 
time made it very difficult for consultants to accurately 
assess all of the ramifications which might aa- or deter the 
hospital from developing a program. Second problem was to 
what extent medical education goes; that is, what is the 
spectrum of possible recommendations. Do they include pro- 
grams for nursing education, have anything to do with inter- 
hospital action or communications, what can be done. At 
this point Dr. Saul distributed a sheet listing a number 
of possible recommendations v;hich the consultants could 
make. This list was by no means to be a definitive list 
of recommendations, nor were all of these recommendations 
expected to be made to any one hospital, at any one time; 
simply a guide list for the consultants. 

Dr. Mitchell said that the consultants should not have 
a rigid format, that not all consultants will be able to 
project the same image to various hospitals. Dr. Saul re- 
plied that there are two explicit aspects of this consultation 
service which the research team needed to know: first, what 
materials go out with the consultants and are given to the 
hospital; second, for the consultants to report precisely 
what was discussed with hospital personnel. The key issue,' 
said Dr. Saul, was not what the consultants should do, but 
simply to report, in fact, what they do do. At this point 
Dr. Kramer suggested a jnodel for research design: one column 
which comprises those items which consultants could recommend 
as an ideal program for a hospital, a second column listing 
those items which are feasible for the hospital to undertake, 
a third column for those items which were actually recommend- 
ed to the hospital, and fourth, a listing of the achieved 
results. 



F-l£ 



During the ensuing discussion, Dr* Saul reiterated that 
recommendations should be sufficiently explicit so that the 
research team will be able to make a degree of implementation 
by the hospital* He said that oftentimes recommendations 
are vague and not clearly defined* 

Questions arose as to the availability of money which 
a hospital might spend: this might be a major factor in hos- 
pitals" iraplfementation of programs* Dr. Stearns mentioned 
that if hospitals cannot afford to develop a program or im- 
plement a program, PMI will find ways to support them, and 
provide sources to which they could turn for financial sup- 
port, for example, the Regional Medical Program* 

The meeting then digressed somewhat to a discussion of 
motivating and developing interests within the community hos- 
pital* Dr. Stearns pointed out that each hospital needed 
an environment which would assist the doctors to want to be- 
come involved in education. Dr. Mitchell mentioned that 
from his experience there are almost always one or two on the 
staff who will attempt to start an educational program going, 
and the consultant can be a catalyst, focal point. But the 
consultant has to sense who these people are, and find a way 
of getting to them on a face to face basis. 

The meeting adjourned for the day at 10:30 p.m. 

On June 28, the meeting commenced at 9:00 in the morning. 
Mr. Pyke briefly discussed the mechanics of the contract: 
such items as cost reimbursment, the need to justify expendi- 
tures w:;' -adequate documentation of expenses, consultation 
fees ana travel expenses. 

Dr. Stearns then continued his discussion of the consul- 
tation process. Considerable time was spent in discussion 
of the consultation process and some of the details in arran- 
ging hospital visits, who the consultants should see in the 
hospital, how should they bill their time, and items of thrs 
nature. Dr. Stearns also discussed in detail the various 
forms which will be used. He mentioned that the consultants 
presence in the hospital means a great deal, but if it's 
going to be difficult for the consultant to elaborate every- 
thing that is going on, which is apparently required, it 
might be useful co have one member of the research team ac- 
company the consultant on visits. This was not looked upon 
favorably by consultants who are new to the Institute, and 
who had not been to the hospitals before. The old consult- 
ants, however, felt that this might be a useful device. It 
was finally agreed that a member of the research team would 
accompany each consultant on one of his initial site visits. 
This person would act merely as an observer and would not in 
any way attempt to influence the consultation process. The 
question arose as to how much interaction the consultant 
should have with the hospital after the cut-off date, October 
31. The concensus was that there should be little interaction 
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by the consultant if -possible* >.CHQw6'weri- the urfeseSteh team 
refcognizedcthat there would be some which is perfectly 
allright as long as there is an active record of what trans- 
pired in each case. 

Meeting adjourned for lunch at 12 noon. 

The meeting convened at 1:15 p.m. Dr. Saul opened the 
f session with a discussion of characteristics of hospitals 

which might be important to know and assess. These charac- 
teristics include age of medical staff, number of board el- 
igible individuals, the years the hospital has existed in the 
community, the nxamber of specialists on the staff, hospitals* 
history of participation in continuing education, its medi- 
cal library and use of journals and publications. Dr. Kramer 
added the ratio of physicians topopulation of the community. 
Dr. Mitchell suggested that an important characteristic 
might be how many staff physicians have used outside con- 
sultants in certain cases. Also mentioned were the kinds 
of committees which exist in hospitals and the type of phys- 
ician who chairs the committee; whether he is a member of 
the old guard and whether there is a strong hierarchy in the 
hospital staff. Distance from Boston again was mentioned as 
were radii from various medical schools, not just the Boston 
medical schools. Dr. Mitchell mentioned the kind of resi- 
dency and intern programs. Dr. Rashin mentioned the presence 
of research projects, and how many of the staff members teach 
elsewhere. Dr. Stearns mentioned the type of control the 
hospital is under, the size of the library, the number of 
journal subscriptions, amount of money available in the hos- 
pital for education. Dr. Mitchell mentioned the qualifica- 
tions or requirements for doctors to become members of an 
active staff. Dr. Kramer suggested the background of the 
hospital administrator might be useful to know. Also the 
presence or absence gf other institutions of education or 
medicine in the community. 

Dr. Saul summarized this particular session of the meet- 
ing by suggesting that the research team would develop a list 
of these independent variables for review. 

At four p.m. the meeting adjourned with the exception 
of staff members and the evaluation consultants. Dr. Saul, 
Dr. Cohen and Dr. Kramer. The evaluation consultants and 
staff discussed the independent variables again, and Dr. 
Kramer suggested that some of the environmental attributes 
such as the community socio-economic status should also 
be considered. He suggested some resource materials which 
the research staff should use. Dr. Kramer also reiterated 
the importance of research personnel accompanying the con- 
sultants on at least one site visit; the staff should de- 
velop a list of behavioral characteristics in addition to 
discussion reported by the consultants to get an objective 
assessment of what actually happens in a consult. 



Next hour was a general discussion of various means 
by v;hich outcomes could be assessed, such as checking at- 
tendance records, interviewing physicians who do partici- 
pate in meetings. Dr. Kramer asked if we wanted any 
before and after measurements .to develop a base line for 
hospital behavior. Dr. Saul felt that this would not be 
required under the design as conceived at thic time. 
Dr. Saul said that we do have some indication of before 
measures resulting from the questionnaires, forms which 
were sent out by the consultants, which do report upon 
the current educational activities of the hospital. We, 
of course, know the information which is available through 
American Hospital Association records and other hospital 
records . 

Meeting adjourned at 5:00 p«m« 
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FORMAT FOR WRITING REPORTS OF VISITS 
TO CONTRACT HOSPITALS 



The following sections should be included in your report of visit: 



1. Title or position of hospital participants in 
the visit 

2. Date of visit 

3. Areas of discussion (who and what was said) 

4. Any suggestions (informal or oral recommenda- 
tions) that were made to the hospital by you 
(and to whom they were addressed in the hos- 
pital) 

5. The social-political atmosphere of the'hospital 

6. The staff's attitudes or feelings toward Post 
Graduate Medical Institute and/or continuing 
education 

7. Any follow-up that would be necessary for Post 
Graduate Medical Institute to do - or you - or 
the hospital as the result of your visit. 



PHASE I 

POSTGRADtTATE MEDICAL INSTITUTE 
AGENDA FOR CONSULTANT MEETING 
September 12, 1967 

I. Regional Medical Programs - Maine ( Tri-State 
II. Summary of visits - Dr. Simmons 
III. Development of Local Education Coordinator for Continuous Use 
IV. Use of Forms and Checklist 
V. Scope of Recommendations 

A. Oral vs . written 

B. Medical vs. educational 

C. Physician vs. para-medical 
Explicitness of recommendations 

VI* Development of Resource Material for Consultants 

A. Community hospital libraries 

B. P.A.S. 

VII* Affects of Observers on Consultation Process 



• 



POSTGRADUATE MEDICAL INSTITUTE 
GUIDE SHEET FOR PREPARATION OF HOSPITAL RECOMMENDATIONS 



The purpose of this guide sheet is to assist you in pre- 
paring your hospital recommendations. This sheet is in no 
wax intended to restrict either the nature or extent of recov- 
aendations you will make. 

All recoataendations* however, must be explicit and must re* 
fleet anx oral recommendations made during the consulting se- 
quence. Recommendations should be considered in terms of 
iaaediate goals, intermediate goals, and long-range goals. 



I. iasic Considerations 

A. Does the hospital have a D.M.E. or local coordinator 
who can be called upon to promote and develop the edu- 
cational program on the .local level? 

i. What are the existing educational activities? 

C. What other activities not considered educational bx 
the hospital max serve education purposes, e.g., 
audits, tissue committee, utilization committee? 

D. How manx local staff physicians are available to 
attend education sessions? (Review staff question- 
naires for D,E, and F.) 

E. How many hours per month are realisticalty available 
, for formal teaching sessions with attendance of so% 

of the staff? 

I 

F. How manx staff physicians are willing to participate 
as instructors, discussions leaders, etc.V 

I 

G. What about education for allied health personnel? 

H. Has the hospital established a realistic budget for 
its educational activities? 

I* What are the goals and objectives for each activUy 
and each participant? 

II. Classes of Recommendations 
— ^ 

Intra-hospital 

X. Group participation, e.g., rouhds, conferences, 

guest lecturers, record review^ current case audit, 
guest resident program. Responsibility for edu- 
cational activities could be disseminated among 
the staff, e.g., over a 10 month basis. 

2* Individual learning experiences, e.g., T.V., radio, 
programmed instruction, library, x-ray of the week, 
pathology sections of the week, films, audio digest, 
patient coverage for outside educational meetings 
and courses . 

Inter-hospital 

Translation of some of these same ideas in II. A into inter 
hospital activities. 
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PROCHDURES AND FORMAT 
FOR TRANSMITTAL OF RECOMMHNDATIONS 



I • Mailing 

a. All recommendations should be mailed through Post 
Graduate Medical Institute 

b. All recommendations should be sent to; 
LOCAL COORDINATOR - original 
HOSPITAL: ADMINISTRATOR - copy 
PRESIDENT OF MEDICAL STAFF - copy 



2 . Format 

a. Initial paragraph should acknowledge and compliment 
persons involved and their activity. 

b. List EXPLICIT (detailed, specific) recommendations 
with explanatory statements accompanying . 

c. Recommendations should be arranged in a time sequence 
(Immediate, Intermediate, and Long Range) encouraging 
implementation of recommendations at earlier dates 
whenever possible. 

d. A concluding statement should thank the hospital and 
indicate your plan or willingness to visit the hos- 
pital for elaboration or clarification of recommenda- 
tion. 
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PHASE I 

POSTGRADUATE MEDICAL INSTITUTE 

DINNER MEETING April 25, 1968 
Hotel Somerset, 5:30 P.M. 

AGENDA 

I. Preliminary Findings of Current Activities 

A. Implementation of Recommendations - Mr. Madden 

B. Utilization of Forms and Local Resources - Miss Kalberer 

C. Procedural Problems - Mrs. Adler, Dr. Saul 
II. Orientation of Contract Extension 

A. Operational Requirements - Mr. Pyke 

B . Research Considerations -Dr . Saul 

C. Consultant Commitments « Dr. Saul, Miss Kalberer 

D. Preparation for May-June Visits - Dr. Saul 

III. Consultant Training and Local Coordinator Training - Dr. Saul 
IV. "The Big Picture" - Dr.- Stearns 
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AGENDA 

TRAINING SESSION FOR PMI MEDICAL EDUCATION CONSULTANTS 
AND COMMUNITY HOSPITAL EDUCATION COORDINATORS 
New Ocean House, Swampscott, Mass. 
June 19, 1968 

8 : 30-12 : 30 Consultants 

12:30-8:30 Consultants § Local Education Coordinators 

TIME S ACTIVITY PARTICIPANTS 
8:30-9: IS 

ORIENTATION TO DAY'S ACTIVITIES Ezra Saul 
9:1S-9:4S 

PHILOSOPHY OF PMI Norman Stearns 

A. Goals of Consultation Process 

1. Stimulate staff to assume 
responsibility is a resource 
for education program 

2 . Develop community hospital 
physi cian education programs 
based on full utilization of 
local resources (including 
manpower and facilities) and 
supplemented by external inputs 
as needed 

3. Use community hospital as a 
major focal point for con- 
tinued medical education for 
M.D. 

B. Common Manifestations of Goal 
Attainment 

1. Appointment of a local coordi- 
nator 

2 . Provision for financial support 
of education program (admin- 
istration, staff, other sources, 
e.g., drug companies , foundations) 

3* Allocation of secretarial assis- 
tance and program publicity 
relations 



TIME S ACTIVITY 



•7 PARTICIPANTS 



4. Full identification of 
local strengths and 
weaknesses 

5. Maximum use of local 
staff as ins true tors 

6. Use of outside expertise 
to complement and supple- 
ment 1 ocal resources 

7. Initiation of regularly, 
scheduled education ac- 
tivities (e.g., grand 
rounds , medical audit, 
CPC, guest lectures, etc.) 

QUESTIONS AND ANSWERS FROM CONSULTANTS 

9:45-12:15 

CONSULTATION PROCEDURES 

9:45-10:15 

A. Establishment of Rapport 



Ezra Saul 5 Medical 
Ed . Consultants 



10:15-11 :00 

B. Collection and Use of Data 

11:00-11:30 

C. Recruitment of Local Parti- 
cipation 

"Force Field Analysis" 
11 :30-12:15 

D. Construction § Transmittal 
of Recommendat ions 

1. 1967-1968 types & numbers 

2. Cor ^ '-recommendations 

3. "Sensitive" recommenda- 
tions 

4. Construction of recommen- 
dations 

5 . Report writing 

6. Review of '69- '69 time 



Norman Stearns § 
Med . Ed . Consultants 



Ezra Saul 



Marge Kalberer 

Norman Stearns 

Kathy Adler 5 
Norman Stearns 



Harold Pyke 



F-4c 



TIME § ACTIVITY 
12:15-12:30 

FUTURE TRAINING SESSIONS S 
MIDDAY EVALUATION 

12:30-2:00 

LUNCHEON Priscilla Room 
Consultants and Local Coordinators 

2:00-2:15 

WELCOME AND ORIENTATION TO AFTER- 
NOON AND EVENING SCHEDULE 
2:15-2:45 

REVIEW OF PMI PHILOSOPHY, 
PROGRAM AND CONSULTANTS 

FUNCTIONS OF THE LOCAL 
COORDINATOR 

A. Data Collection and Analysis 

B . Program Production 

C. Program Assessment 

2;45-4:15 

PRESENTATION OF 4 RESUMES OF 
HOSPITAL EXPERIENCES 1967-1968 

A. Establishment of Rapport 

B . Data Col lection 

C . Recruitment of Local* 
Participants 



PARTICIPANTS 



Norman Stearns 
Marge Kalberer 



Ezra Saul 

Norman Stearns 
Harold Pyke 



Consul tanf* S 
Local Coordinators 

Dr. Madoff 

Dr. Uhl 

Dr. Robertson 



D. Transmittal of Recommendations Dr. McCombs 
4:15-5:00 

A. Hospital as a Social System 

B. Library Project 



C. Regional Medical Programs 
PL-89-239 

D. Comprehensive Health Planning 
PL-89-749 



Ezra Saul 
Wendy Ratcliff 
Norman Stearns 



Norman Stearns 
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TIME S ACTIVITY 
S:00-S:1S 

EVALUATION OF DAY 
(fo early departees) 

5:15-6:1S 

cocktails" 

6:15-7:30 

DINNER 

7:30-8:30* 

GROUP DISCUSSION 
Question-answer period for 
local coordinators 

EVALUATION OF DAY 



PARTICIPANTS 



Ezra Saul 
Robert G >» 



Local Coordinators 

Consultants 

^ PMI Staff 

Ezra Saul 
Robert Gold 
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PHASE II 



Postgraduate Medical Institute 



EVALUATION OF TRAINING SESSION Name: 
June 19, 1968 

Hospital : 
Title: 



I. In my role as a consultant or DME this training session was 

(Circle one) a \.i;ste very 

of time useful 

1 2 3 4 5 



Please explain -/our choice 



II. The most useful part of this training session was: 



III. The least useful part of this training session was: 



IV. What next? 

A. Anticipating my role as a consultant or DME, I need: 

(Circle one) no more S&^h more 

training . '. — training 

1 2 3 4 S 



B. Further training sessions should focus on: 

(Please state your choices in order of preference) 

1 ._ - ■ ■ - 

2. — 

3. , — 

4. 



If you were in charge of this training session how would you 
have altered it? ^ - 
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VI, Please give additional comment on the training session, 
including techniques and method used, length of time, 
location and type of facilities, effectiveness of train- 
ing staff members, etc. 
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PHASE II 

POSTGRADUATE MEDICAL INSTITUTE 

CONSULTANTS' MEETING 

September 28 , 1968 

9 : 30 a,m. 

Holiday Inn of Kaltham 

Totten Pond Road at Rte, 128 
(Winter Street Exit 48 & 48E) 



I. The Consultant and 

II • The Consultant and 

III, The Consultant and 

IV, The Consultant and 

V, The Consultant and 

IV, The Consultant and 



MORNING: 

PMI 

His Hospital 

Allied Health Resources 

AFTERNOON: 
Educational Aids 
Evaluation 
Time 



PHASE II 
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Postgraduate Medical Institute 



EVALUATION OF CONSULTANTS' MEETING 

September 28, 1968 NAME 



I. In your role as a consultant how useful was this training 

session? * (circle one number) 

a waste of time very useful 



12 3 

Please explain your choice 



II. The most useful part of this training session was: 



III. The least useful part of this training session was: 



IV. To what extent do you feel you need further training as a 

consultant? (circle one number) 

no more much more 

t raining t raining 

1 2 3 4 5 

V . What issues or areas would you like to focus on- at future training 
sessions? 

1 . 

2. 

3 . 

4 ._ 

5 . 

6. 



ADDITIONAL COMMENTS : 



PHASE II 
POSTGRADUATH MEDICAL INSTITUTE 
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CONSULTANT - HOSPITAL COORDINATOR MEETING 
November 16, 1968 

AGENDA 

MORNING - PMI CONSULTANTS ONLY 

9:00 Arrivals: Hot coffee and a smile from everyone. 

9:30 Arrivals: Cold coffee and a smile from Marge 

Kalberer. She'll love you anyway. 

9:30-11:30 The Consultant and His Hospital: 

experience, problem and insight sharing. 

(Open di scussion) 

Mode rat or -Marvin Mitchell , M.D. 

Resource Consultant-Elmer Van Egmond, Ph,D, 
Resource Staff -Norman Stearns , M,D. 

Ezra Saul, Ph.D. 

PMI Staff 

11:30«-12:00 Assessment of Morning Session 

AFTERNOON - LOCAL HOSPITAL COORDINATORS AND CONSULTANTS 
11:30-11:45 Registration of Local Coordinators 

12: 00-1 :20 Lunch 

1:30-2:00 PMI Core Library Development Program- 

Wendy Rat cl iff. Program Director 
Charles Colby, Administrator of 
.Countway Library of Medicine 

(Discussion and Ques t ions) 

2:00 The Use of Audio Visuals in Continuing 

Medical Education 

Harold Pyke , Director 

Continuing Education, Tufts 
Dental School 

2:30-3:00 The Physician Learner: Principles of 

Continuing Education 
Ezra Saul, Ph.D. 



AGENDA 



Page 2 



3:00-3:45 Workshop: Application of Educational 

Resources and Principles to the 
Community Hospital Setting 

Moderators - Elmer Van E<?inond, Ph.D. 
Ezra Saul, Th.D. 

3:45-4:00 Coffee Break 

4:00-4:45 Workshops - (regrouped and continued) 

4:45-5:00 Assessment of Afternoon Session 

5:00-6: 00 Cocktai? s 

6: 00-7 :30 Dinner 

7:30-9:00 TheLocalCoordinatorandHis 

Hospital : E xperience , P rob lem and 
Insight Sharing. 

(Open discussion) 

Moderator - Elmer Van Egmond, Ph.D. 
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PHASE II 
POSTGRADUATE MEDICAL INSTITUTE 
ASSESSMENT OF CONSULTANT-HOSPITAL COORDINATOR MEETING 

November 16, 1968 

NAME^ 

HOSPITAL 



I Please cite and explain one instance of personal learning for 
you today vr.ich you think you will attempt to apply in your 
capacity as consultant or local coordinator. 



II. The most useful part of today's meeting was: (explain) 



III. The least useful part of today's meeting was: (explain) 



IV. What areas or topics would you like to focus on at future 
meetings ? 

1 . . — 
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PHASE II 
POSTGRADUATE MEDICAL INSTITUTE 
Consultant - Local Coordinator Meeting 
February 1, 1969 
Sheraton Motor Inn - Lexington, Massachusetts 
REVISED _A G ENDA 

Participants 

9:00 - 9:30 a.m. Implications of Current 

Project - Local - National Norman Stearns, M.D. 

9:30 - 10:30 Principles of Educational 

Evaluation Ezra Saul, Ph.D. 

10:30 - 11:00 Medical Audit Techniques Victor Sidel , M.D. 

11:00 - 12:00 Designing Continuing Medical Clement Brown, M.D. 

Education Programs Daniel Fleisher, M.D 

12:00 - 1:00 p.m. Lunch 

1:00 - 3:00 Designing Continuing Medical Clement Brown, M.D. 

Education Programs - continued Daniel Fleisher, M.D 

3:00-- 3:30 Required Resources for 

Educational Evaluation Ezra Saul, Ph.D. 

3:30-4: 15 Discussion of Local 

Problems Ezra Saul, Ph.D. 

4:15 - 4:30 Evaluation of Day's 

Meeting 
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PHASE II 
POSTGRADUATE MEDICAL INSTITUTE 
Evaluat ion 

CONSULTANT - HOSPITAL COORDINATOR MEETING 
February 1, 1969 

Name 

Hospital 



I. Please circle the number which best reflects your evaluation of 
today's meeting. 

not very 

useful . "seful . 

"1 2 3 4 5 

II. The most useful part of today's meeting was: (explain) 



III. The least useful part of today's meeting was: (explain) 



IV Please cite and explain one instance of personal learning for you 
today which you think you will attempt to apply in your capacity 
as consultant or local coordinator. 



ERIC 



V . 



What areas or topics would you like to focus on 
(please indicate your choices in order of preference.) 



at future meetings' 



1, 

2. 

3, 

4, 

5 

6 



p 



PHASE II 
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POSTGRADUATE MEDICAL INSTITUTE 



HOSPITAL MEDICAL EDUCATION COORDINATORS MEETINC 

April 26, 1969 

Sheraton Rolling Green Motor Inn - Route 93 
Andover , Massachusetts 

9:30 a.m. - 4 :00 p.m. 



Morning Session 

Moderator : Edward Haml in , M . D . 

Director of Medical Education 

St . Luke • s Hospital 

New Bedford, Massachusetts 

9:30-11:30 RESOURCES FOR CONTINUING 

MEDICAL EDUCATION 

Postgraduate Medical Institute Marvin L. Mitchell, M.D. 

" ~ PMI Program Director 

Medical Schools of the Region 
Tufts University 



Dartmouth College 



Boston University 



Harvard University 



George Robertson, M.D. 
Assistant Dean for 
Continuing Medical 
Education 

Dean Seibert, M.D. 
Assistant Dean for 
Regional Affairs 

Thomas Dawber, M.D. 
Program Planning 
Officer, B.U. Medical 
Center; Associate 
Professor of Medicine 
B.U. School of Medicine 

Daniel Federman, M.D. 
Assistant Dean for 
Cont inuing Educat ion 



Brown University 



(Speaker to be announcec 



Page 2 
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Regional Medical Programs 

Tri-State Regional 
Medical Program 

Maine Regional Medical 

11:30-12:15 PROFESSIONAL ACTIVITIES STUDY 

Use of PAS-MAP to Document 
Educational Needs ■ 

LUNCHEON BREAK 

Afternoon Session 



12:30-1 :30 



1:30-2: 15 



2:15-4:00 



Leona Baumgartncr, M.D 
Executive Director 

Manu Chatterjee, M.D. 
Program Coordinator 



Kenneth Teich, M.D. 
Ann Arbor, Michigan 



4:00 p.m 



Moderator: Robert P. McCombs , M.D. 

President, Postgraduate 
Medical Institute 

THE PROBLEM ORIENTED MEDICAL John Bjorn, M.D. 
RECORD AS A BASIS FOR Hampden, Maine 

PHYSICIAN EDUCATION 

BRIEF PRESENTATIONS AND 
DISCUSSIONS OF "REAL-LIFE" 
HOSPITAL EDUCATION ACTIVI- 
TIES AND TECHNIQUES BY 
HOSPITAL COORDINATORS 

(Specific speakers to be announced 
with open discussion by all present) 

ADJOURNMENT 
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»sn II 

POSTGRADUATE MEDIC\L INSTITUTE 
Evaluat^ion 
HOSPITAL EDUCATION COORDINATOR MEETING 
April 26, 1969 



Name 



Hospital 



I. Please circle the number which best reflects your evaluation of 
April 26th meeting. 

not useful ve ry uycful 

1 2 3 4 5 

II. The most useful part of today's meeting was: (explain) 



III. The least useful part of today's meeting was: (explain) 



IV. Please cite and explain one instance of persojjal .learning for you 
today which you think you will attempt to apply in your capacity 
as consultant or local coordinator. 



V. What areas or topics would you like to focus on at future meetings? 
(Please indicate your choice in order of preference.) 



1 
I 

erJc 



PHASE II 
PMI CONSULTANT DINNER MEETING 



57 Carver Street Restaurant 
June 10, 1969 
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Agenda 

5:30-6:30 Cocktails 

6:30-J0:00 Dinner ar.d Business Meeting 

(Moderator: Dr. Norman Stearns) 

I. Fjn^l Hospital Visits 

Handouts: (1) Guide for final visit 

(2) Sumnary sheets on hospital's 

willingness to continue consultation. 

II. Continuation of Consultation Program 

III. Reccrxennations for Future Consultation 

Discussion vill focus on the following questions: 

A. Khat changes should be made in the consultation 
procedure, e.g., hospital preparation, hospital 
com-itracnt, consultees, data collection procedure, 
reporting infornation to PMI, etc. 

B. What is "good" consultation in terms of 
intensity and input from the hospital ' s point 
of view? 

What is "good" consultation in terms of 

intensity and input from the consul t -.nt ' s point 
of view? 

C. HoK should the consultant be prepared for his role? 

D. What constitutes a likely hospital candidate for 
successful consultation, i.e., what conditions 
are more likely to exist if a hospital is to 
effectively utilize the consultation? 

E. Should there be a package of complementary 
services accor.nany ing the consultation, i.e., 
allied health,' nursing, library or other types 
of consultation services? 
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SUGGES-^HD CONSULTATION PROCEDURE FOR FINAL VISIT 



I. Visits should be made during the summer of 1969 if possible. 
(Prior to the onset of the fall education program.) 

II. Appc.ntments and arrangements for the visit should be made by 
the consultant with the hospital. Dates of the visit should 
be forwarded to Marge Gctchell at PMI prior to the visit. 

III. Suggested areas of discussion during the visit: 

A. Current state of education activities and local 
staff involvement. 

1. Quality of present education activities. Should or can 
they be improved? Should they be eliminated or modified? 
(Check each education category, e.g., grand rounds, 

CPC, lectures, etc. ) 

2. Involvement of staff in current education activities. 
Has there been full or anywhere near full participation 
by the staff in direct teaching activities or in 
arranging for education activities? Who has been 
coming to the sessions? What has the attendai^ce been? 

3. Use of open discussion in education sessions. What 
§pecific education activities involve open discussion 
of current cases by several staff physicians, e.i,., 
grand rounds, audits? 

B. Identification of education needs. 

1. Has any attempt been made to design education sessions 
relating to felt or otherwise identified needs, e.g., 
using PAS documented needs or tis^ae committee reports, 
etc.? 

2. Would the DME or local coordinator be willing to survey 
the staff with regard to their simple expression of 
what is needed to improve care that could be fulfilled 
by some form of education activity for physicians 

(and possibly for nurses or other allied health pro- 
fessionals working in the hospital?) 

3. Has any attempt been made to encourage development 
where indicated of local expertise by assignment of 
responsibility for patient care and education in 
specialty areas of medical practice to local staff 
members? Can this be done? 

Has any attempt been made to send such individuals 
to medical centers for special training in order to 
fill local needs in terms of pdticnt care or education. 
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C. Program goals and objectives for 1969-70. 

(Hospital should be encouraged to state these 
objectives in writing.) 

1. What is needed and what plans have already been 
made to complete the stated goals? 

2. What are possible sources of help outside of the 
hospital for fulfilling goals, e.g., PMI, RMP, 
medical schools, voluntary agencies, etc. 



IV 



Consultant should make a final report indicating his opinion 
of the hospital's progress over the past two years and noting 
any final recommendations for program improvement. 
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Postgra d u at e Medical iTi stitirte^ 
PHASE II 

PUS CONTRACT - INTERVIEW MEETING 
April 26, 1969 
Saxon Room 
Sheraton Rolling Green Motor Inn 
AGENDA 

1. What is PMI? (handout of activity sheet) Marge 

1 1 . Historical S Technical SumTna ix^gl 

Research Plan What are we going to do with 
data? 

III. Interviewing Situations and Problems 

A. Catholic Hospitals 

B. Osteopathic Hospitals 

1 . Terminology 

2. Dependency and def ensi veness 

C. Group Interview 

D. Impatient Physician or Interrupted 
Interviewer 

E. Control Hospitals 

1. Need to clarify project and PMI 

2. Avoid specific future commitment 
to PMI 

F. Hanv'ling hostility - lack of recommenda 
tions by consultants, etc. 

G. RMT relationship - refer questions 

IV. Review sample packets Kathy 5 Group 

A. Arrangement - who to call, when, etc. 

B. Interview - who to see, etc. 

C . Written Recommendations 1967 -6S 

D. Written Recommendat 10*^5 1968-69 

E. Missing information - to be picked up 

1. Hospital Attributes 

2. 1968-69 Education Activities Schedules 

F. Field Training and Tentative Interview Schecules 

g! Hospital Questions - Unable to irterview all people, 

hospital which received no 
recommendations , etc . 



Kathy 
Marge 

Ezra 

Bob Gold 
Dick Carter 

All 



V. Reprints on continuing education, hospital libraries 
and Osteopathic Hospitals 

VI. Administrative Details Dick Carter 

A. Expenses 

B. Voucher form 

C. Charge cards for auto and phone 
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CASE STUDY 
Hospital A 



Hospital A is a 167-bed, non-profit, community gen- 
eral hospital serving two adjacent suburbs of a large 
metropolitan city. Although it is the only hospital ex- 
isting in these communities, the staff and potential pa- 
tients of Hospital A have access to major medical centers 
11 miles away. Hospital A is accredited by the Joint 
Commission on Accreditation of Hospitals and Blue Cross. 

Prior to PMI consultation, September 1967, Hospital 
A operated with facilities and services comparable to a 
hospital of similar bedsize, e.g., blood bank, clinical 
lab, operating room, delivery room and organized hospital 
auxiliarv. Facilities and services did not include an 
intensive care unit, a coronary care unit, an outpatient 
department or a home care program. 

The active medical staff numbered 55, 28"o being gen- 
eral practitioners; 71% specialists. The mean age was 
52. Eighty-three percent of the staff were affiliated 
with other hospitals . The radiologist, pathologist, 
chief of medicine and chief of surgery were not hospital- 
salaried. 

The hospital regularly scheduled committee meetings 
which dealt primarily with administrative affairs. The 
Credentials Committee met quarterly (January, April, June, 
October''; the Executive, Tissue, Medical Records, Utili- 
zation Review, Pharmacy, Medicare, Library, Prenatal 
Mortality, Infections, Procedure, and Standardization 
Committees met monthly. 

Hospital A also held departmental meetings in order 
to fulfill accreditation requirements. The departments 
of Surgery, Medicine, Pediatrics and Ob/Gyn met separately, 
once a month, to review current clinical cases on their 
services. Those meetings were largely clinical record 
reviews, with the major effort directed at quantity rather 
than a qualitative educational purpose. The hospital was 
under the impression that the Accreditation Board dis- 
allowed record reviews which primarily served educational 
functions. The Medical meetings, however, did include 
an occasional guest speaker and/or attending staff dis- 
cussion . 



Continuing medical education activities during 
1966-67 included: four meetings covering topics of wide 
general interest and bi-monthly evening meetings of sur- 
geons which dealt with criticism of patient ther'apy. 
Inter-staff consultations occurred frequently. Hospital 
activities did not include grand rounds, bedside rounds, 
x-ray conferences or clinical pathological conferences. 
Prior to 1966, the medical staff attempted to use in- 
patients for teaching rounds , but the conferences atrophied 
because of limited support. 

Regarding a structure for continuing medical education 
activities. Hospital A had an Education Committee, but 
no appointed Director of Medical Education. A lecture 
hall with projection equipment was furnished, but no 
budget funds for educational programs and no coverage 
for doctors who might wish to attend such meetings were 
provided. Physicians and other staff members had access 
to a library but the library was not up-to-date and only 
subscribed to 3 journals. The hospital offered in-service 
programs for nurses and other allied health professionals 
but physician interest in these programs was not sig- 
nificant. Seventy-five percent of the medical staff 
specialists attended rounds, clinics or other educational 
activities at nearby university hospitals, yet Hospital 
A had no established education program in conjunction 
with other hospitals. 

Early in September 1967, a trained PMI consultant 
visited Hospital A to assess the educational activities 
offered at the hospital and to recommend changes in and/ 
or additional activities. The consultant met with the 
Assistant Administrator, Chief of Surgery, Chief of Medi- 
cine, Director of Nursing, two members of the Education 
Committee and several other representatives of the medi- 
cal staff. 

During this initial visit, the consultant left ques- 
tionnaires for each medical staff member for the purpose 
of collecting data to aid in program development at the 
hospital. A summary of that data revealed that 78% of 
the staff were willing to attend teaching sessions, 33% 
to instruct, 45% to be taught by their own hospital 
staff members and 48% to use their own patients in 
teaching. This summary indicated substantial staff in- 
terest in development of an education program at Hospital 
A. However, noting the high percentage of doctors par- 
ticipating in continuing education programs elsewhere, 
the consultant felt there might be a reluctance to par- 
ticipate at one instituted by Hospital A. 

Following his first visit, the consultant sent a 
series of recommendations to the hospital for educational 
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program development. His recommendations were as follows: 

1. The medical staff should have the program 
organized by a Director of Medical Education 
(DME) . 

2. Hospital A should provide adequate secretarial 
help to facilitate the organization and smooth 
development of the program. 

3. Viewed on a monthly basis, the program should 
include the foil owing: 

a. One full staff conference oriented to areas 
of mutual interest 

Ten such conferences should oc cur 
yearly , omit ting Ju ly and August . The res - 
ponsibility for the programs should be ro- ^ 
t ajted through various services, i.e.. 
Medicine , Surgery , Pediatrics, Obstetrics, 
Gynecology, Urology, Orthopedics, Radiology, 
Pathology, Neurology, Dermatology . 

b. Service meetings as occur now for Medicine, 
Surgery, Ob/Gyn. and Pediatrics 

In addition to the coverage of routine 
audit material, these meetings should serve 
an educational function. Individuals on 
each service should rotate responsibility 
for preparing these meetings in terms of 
presentation, obtaining guest speakers, 
etc. Encourage your radiologist and path- 
ologist to contribute to these meetings. 
All staff members should recognize that no 
inconsistency exists between a successful 
medical audit and a meaningful educ at ion a 1 
activity. 

c. Service-Patient Rounds - - 

The staff should see and discuss in- 
teresting problems in the hospital. Organize 
these rounds by services . Depending upon 
the attendance , service rounds should be con- 
ducted at the bedside or in a conference room 
with the patients brought to th**e group. Again, 
the staff should rotate responsibility for* 
conducting these rounds. The rounds should 
consist of a review of the pertinent clinical 
problems with some discussion relating to patho- 
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genesis, diagnosis or therapy. Group participation 
should be encouraged during the rounds. 



d. One additional service meeting, the format as 
yet to be determined 

Proposals for this meeting include a journal 
club, a clinical pathology conference, a review 
of deaths witli presentation of pathology, and 
additional patient rounds. Perhaps each service 
should decide what would be most appropriate for 
this meeting. Indeed, a service might wish to 
vary the type of meeting initially so as to find 
out what is most successful. 

4. Sessions described above should include the following: 

a. Kinescope material from "Boston Medical Reports" 

The staff should show available reports at 
luncheon meetings . Th "^y could probably obtain 
the luncheon room and perhaps the Administration 
would provide meals for those at ten ding. When 
used, these kinescopes will provide an introduc- 
tion to further discussion by the physicians pre- 
sent. 

b. Ten to twelve speakers per year 

Speakers should receive an average s t ipend 
of $50. The hospital in conjunction with the 
medical staff should pay this stipend. 

5. In preparation for the Coronary Care Unit, one or 
more members of the Nursing Staff should enroll 
in a Coronary Care Monitor course within the next 
six months. 

6. The hospital should accelerate library development, and 
attempt a goal of eight j ournals and fifteen new 
volumes per year. 

7. Trustees of the hospital should receive a full report 
of the activities that have occurred to date regarding 
the Continuing Medical Education Program; should seek 
financial support at this time to support the pro- 
gram to cover guest speakers, stipends, educational 
materials, library development; and should consider 
seeking a token remuneration , i.e. a paid trip to a 
professional meeting, for the appointed Directors of 
Medical E ducat ion. 



Four months later, PMI sent a professional interviewer to 
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check the progress of the consultant's recommendat ioas * 
The interviewer spoke with the Administrator, President 
of the Medical Staff, Surgical DME, Medical DME , and 
the Director of Nurses. The following recommendations 
had been implemented : 

1. The hospital appointed both a Medical and 

a Surgical DME to coordinate the Continuing 
Medical Education Program. 

2. Secretarial help w.as provided by the administration. 

3 . The topics presented at quarterly staff meetings 
were broadened in order to interest as many of 
the medical staff as possible. Responsibility 
for the programs was rotated through various 
services . 

4. The Medical DME attempted to change the monthly 
service meetings to include an educational function 
in addition to the coverage of routine audit 
material. Each service rotated responsibility 

for these meetings. The pathologist and radio- 
logist were encouraged to participate. 

5. Both the Medical and the Surgical departments 
changed their monthly grand rounds to weekly 
grand rounds . 

6. The hospital presented 10-12 guest speakers 
throughout the year. A pharmaceutical company 
offered funds for guest speakers to the hospital 
if the company approved of the speakers 60 days 
in advance of the program. 

The PMI consultant visited Hospital A again, May 
1968, to discuss further plans for the hospital's continuing 
medical education program with the administrator and the 
Medical D.ME. The consultant suggested that the Medical 
Education Committee invite a nurse to attend their meetings 
and that the hospital administration appoint a librarian 
other than the Medical DME. The consultant also offered 
other recommendations which he confirmed by letter in 
September. His recommendations were as follows: 

1. The core of the Continuing Education Program 
will consist of meetings carried over from the 
previous year with minor alterations. 
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a. The medical service should meet monthl/j at 
which time a guest speaker should discuss a 
medical topic. There should also be an 
occasional case presentation. Res pons ib i 1 i ty 
for the development of these sessions should be 
rotated among the medical staff. 

b. Medical rounds should be held monthly. The 
radiologist and pathologist should be persuaded 
to attend these sessions. 



c. The death meetings for the surgical service 
should occur ' onthly .Discussions should con- 
centrate on management of patients and what 
might have been done to have improved 

out comes . 

d. Surgical service rounds should occur monthly. 
Responsibility for devel oping the format 

for these sessions should be rotated among the 
surgical staff. 

e. The evening surgical meeting should occur bi- 
monthly. These sessions should be regarded as 
a rather "knock -down -drag -out encounter'* in 
which service problems are aired. 

f . There should be a prog^ am of four evening 
meetings for the entire hospital staff with 
guest speakers . 

g. The Ob-Gyn and Pediatric meetit I should occur 
monthly . 

h. The monthly medi cal -surgi cal meeting should be 
di scont inued because of the poor attendance and 
the difficulty in finding suitable topics that 
interest both groups. 

2. Two new programs should be developed during the year 
1968 - 69: 

a. A medical death round should occur monthly, at, 
whicli time two or three interesting deaths should 
be reviewed. The pathologist sliould attend these 
sessions and the Medical DME sliould devel op the format. 



b. An in-service training program for nurses should 
occur weekly with contributions shared by the 
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medical and surgical services. The program 
should be developed in conjunction with the 
Director of Nurses. 

3. Accreditation of educational meetings by the 
Academy of General Practice should be sought. 

It should be cl arly understood and generally pub- 
licized that this type of accreditation might be 
most desirable, as governmental and other influ- 
ences may pass various sanctions regarding phy- 
sicians' activities as based on their educational 
and practice background* 

4. The Administrator's secretary should send the 
hospital's 1968-69 calendar of events to the con- 
sul tant • 

5. Both DME'^s should note which staff members support 
educational programs at the hospital in order to 
evaluate the extent to which the staff can share 
the deve lopment of the educational program . 

6. The hospital should provide honoraria for all guest 
speakers • Last year, a pharmaceutical company 
provided honoraria for the evening speakers. Day- 
time speakers should also receive an honoraria of 
no less than $50. Seek funding, namely $1,000, 
from the trustees. 

7. The librarian should work with the Medical DME to 
weed out unused journals and texts from the li- 
brary and fill the library with appropriate and 
up-to-date material. For this purpose, PMI will 
send a list of suggested material. The admini- 
strator should also provide the librarian with any 
material that he has on library development. Five 
hundred dollars is needed to expand and improve the 
library. Request this funding from the trustees 
and supplement their contribution with staff dues. 

8. If the pathologist needs photographic equipment 
to take pictures of microscopic slides for teach- 
ing purposes, request funds from the trustees. 

9. In recognition of their sewices relating to the 
education program, both DME's should receive funds 
from the hospital for a trip to an educational 
meeting of their choice. 

10. Both DME's should draw up and submit a formal re- 
quest for all funding, except the latter, to the 
Board of Trustees for review and acceptance. In 
addition, they should present the program to the 
Board. If so desired, the consultant will attend 
the meeting to stress the importance of the pro- 
gram and the participation of PMI in educational 
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program planning at the hospital. 

11, The hospital staff should pay particular atten- 
tion to topics pertaining to the new intensive 
care unit, e.g., critical care problems, tech- 
niques and concepts. 

In October, the consultant visited Hospital A to 
attend the first meeting in a series of newly organized 
death conferences. The purpose of these sessions was to 
present relevant and interesting cases. Attendance at 
this meeting was poor. Besides the consultant, only 
eight other doctors attended: Hospital A's two DME's, 
the Pathologist and his assistant, the Chief of Medicine, 
and the Chief of Surgery, one internist and one other sur- 
geon. The doctors who cared for the two patients whose 
cases were presented at the conference did not attend. 
This poor attendance was not attributable to poor pub- 
licity for the Medical DME had publicized the conference 
via weekly newsletters, a monthly newsletter, a black 
board specially obtained for publication of educational 
programs, and an announcement made over the loudspeaker 
immediately preceeding the conference. No clinician 
discussed the cases. The. pathologist had slides pre- 
pared on the cases, but no projector to show then. Instead, 
he used a microscope through which no one bothered to look. 
Although the conference was nearly disasterous, the Medical 
DME decided to continue with the series. It was the con- 
sultant's feeling that the Medical DME lacked the dynamism 
to stimulate the apathetic medical staff towards an active 
participation in this kind of an activity. 

During his visit, the consultant inquired about ^ 
acquisition of funds to support the education program, .-fo 
date the Board of Trustees had offered $500 for library 
development, but rejected the proposal for $1,000 to sub- 
sidize daytime speakers. The Board felt that the staff 
should aid in funding its own education; accordingly, the 
staff voted to pay $10 per man for the educational fund. 

The consultant also learned that the hospital had 
appointed a librarian to relieve the Medical DME of library 
duties. He assured the new librarian of I'MI's assistance 
in setting up an adequate collection for the medical staff 
and advised hin concerning efficient management and main- 
tenance of a hospital library. . . 

Three months later, January 1969, the consultant visi- 
ted Hospital A to attend ojie of the bi-monthly meetings 
of the surgical staff. The meeting was well attended, 
the cases well presented and the discussion afterwards well 



handled. The surgeons, howcvsr. -^n such meetings 

prior to PMI consultation. \ ppointracnt noted 

by the consultant was that onf t v ^e presenter* 
came totally unprcpared"to r subject he had volunr 

tecred to discuss. 

After the meeting, the consultant learned that the 
Medical DKE had cancelled the death onferenco and sub- 
stituted a staff meeting on the t F--d?.y of each 
month, alternately. attended by thi ogist and 
radiologist. He also learned that fitical care unit 
was close to opening and suggested ^ the Medical DME 
initiate an in-service training program to coincide 
with the opening of the unit. 

In April, the consultant re-visited Hospital A to 
attend one of the staff meetings which the Medical DME 
had scheduled in place of the death conference. Again, 
the meeting was well planned and presented, but it was 
not well attended. The speaker was a young internist 
whom the staff described as the "great white hope" of 
Hospital A. The case he delivered was a misdiagnosis, 
a rare but needed presentation. The presentation did not 
include projection of microscopic slides on the case 
(as the trustees had turned down the proposal to provide 
necessary photographic equipment) and no dialogue be- 
tv» ^n speaker and attending physicians followed the 
lecture. 

While attending the meeting, the consultant learned 
that the hospital had held six evening nurse education 
sessions on coronary care. Over 100 nurses from the 
surrounding communities ha. attended each of the sessions 
which included lectures, panel discussions and demon- 
strations. The consultant felt that this venture was 
probably the most successful educational enterprise 
undertaken by Hospital A sii.ce his consultation had begun 
in September 1967. 

The following month. May 1969, PMI sent 2 PMI- 
trained interviewers to check the implementation rate of 
the consultant's second set of recommendations. The 
interviewers spoke with the Administrator, Medical DME, 
Chief of the Medical and the Assistant Director of Nur- 
sing. As recommended the following changes had occurred: 

1. The DME's discontinued the medical-surgical 
meetings . 

2. The Administrator's secretary sent Hospital A's 
1968-69 calendar of events to the consultant. 



3. The DME's noted which staff members supported 

the educationil programs presented by the hospital 
and asked the pathologist and radiologist to 
participate more fully in all education meetings. 

4. The staff and trustees provided over $500 for 
library development. The librarian discarded old 
library books, painted the outside of the library 
books to prevent theft, and announced to the 
staff the purchase of new books. 

5. The DME's submitted a forr^al request for all fund- 
ing, except the stipend for their own services, 

to the Board of Trustees for its review and 
accept ance • - — 

6. The Education Committee sponsored nurse education 
lectures on coronary care. 

7. The DME's publicized the education programs 
offered at the hospital via newsletter and 
bulletin board. 

8. The hospital acquired projector and slide equip- 
ment for the education conferences . 

The consultant visited Ho pital A for the last time 
in August 1969. He reviewed the Hospital's education program 
for the past year and plans for the upcoming year with 
the Administrator and the Medical DME . On paper the program 
appeared comprehensive and wel 1 -planned . Essentially, 
Hospital A had developed their library f aci lit ies and a 
core program consisting of three educational sessions 
per month: a surgical meeting, a medical meeting, and 
Saturday morndng medical rounds. The trustees had funded 
the library development program and the staff had established 
an educational fund for stipends to attending guest speakers. 
All efforts at a pathology, x-ray or death conference had 
failed. The nurse education lectures on coronary care 
weie successful in involving nurse interest, but the hos- 
pital had not established a continuous and integrated in- 
service training program. Concerning future plans for 
educational programs at Hospital A, the consultant felt 
that neither the DME • s nor the general staff had enough 
initiative or desire to add to the existing programs. ^ 

Hospital A had received consultation from PMI regard- 
ing education programs for nearly two years: September 
1967 through August 1969. A trained consultant had visited 



the hospital six times. Twice during his visits he had 
o'^fered recommendations to improve its continuing medi- 
cal education program. Twice he had confirmed these 
recommendations by mail. 

Hospital A did follow through on some of these 
recommendations. The hospital had ; pointed two DME's 
to coordinate its continuing medical education program. 
It had added educational meetings, changed previously 
scheduled meetings to follow an educational format, assigned 
a staff phvsician as librarian and improved both the appear- 
ance and content of the library. Moreover, they had ac- 
quired lirzited funding for these activities from the 
trustees and staff members as well as a pharmaceutical 

company. , , 

In October 1968, Hospital A had entered the PMI 
library program, an additional result of PMI consultative 
efforts A trainee was sent to attend the Library Training 
Institute held in Boston, February 2-7, 1969, sponsored 
by PMI-XERMLS (New England Regional Medical Library 
Service). The hospital also began acquisition of the Medi- 
cal Core Library developed by PMI for physician practitioners 
at community hospitals. 

And vet, in spite of this progression in continuing 
education', the consultant felt that the hospital would not 
further develop its education program. His conclusions 
were partially attributed to the following conditions: 

1. Disinterested chiefs 

Both the Chief of Surgery and the Chief 
of Medicine showed only mild interest in con- 
tinuing education. The chiefs of staff did very 
little to support continuing education in his 
dealings with the trustees. Until chiefs 
of service with a more fervent interest in con- 
tinuing education j orn"Th"e hospital staff, pro- 
gress at the hospital is limited. 

2. Multiple appointments of staff -- 

Many staff members were committed not only 
to Hospital A but also to other hospitals. In 
looking ahead Jtoward eventual regional ization , 
Hospital A nbeds more than three poorly attended 
educational meetings per month to be considered 
as a part of any . intcr-hospital program. 



3. Failure of trustees to provide sufficient funds 

Besides their interest in and initial funding 
of the library development pro j ect , Hospital A* s 
trustees knew virtually nothing about and con- 
tributed nothing to the Continuing Medical Edu- 
cation Program at the hospital. Both the Medi- 
cal DME and the Administrator should inform the 
Board about funds which the trustees of other 
hospitals allot for their hospital's educational 
program. 

In his final report on liospital A, the consultant 
wrote: "I cannot help but re-express my disappointment 
at the inability of the consultation process to have any 
rc . 1 impact that I could identify. It is quite possible 
that the (PMI) research staff will be able to document 
certain changes in activities at the Hospital. Khatever 
may be tabulated on the positive side, however, falls 
vei-y short of the real problems. Until forceful and dynamic 
leaders are placed in appropriate sit ions .. .rather 
little can be expected." 



CASE STUDY 
Hospital B 



Hospital B is a 70-bed, municipally controlled, 
community general hospital. Located in an isolated rural 
section of New England (300 miles from the nearest medi- 
cal center), the hospital serves the city and surrounding 
community. The hospital also influences a 60 square mile 
area, triangularly encompassed by itself and two other 
hospitals. One of the state*s colleges is situated only 
a few miles away. The nearest medical school is over 
400 miles south. 

Prior to PMI consultation, August 1967, Hospital B 
operated with facilities and services comparable to a hos- 
pital of similar bedsize, e.g., clinical/pathology labs, 
operating room, out-patient department. Facilities and 
services included only a partial^ciHE^onary care unit and 
x-ray department and did not include an intensive care 
unit, progressive care unit, social work department or 
rehabilitation in-patient department . 

Eleven physicians (9 of whom were specialists) 
constituted the active medical staff. The mean age of the 
medical staff was forty-six. The hospital had no salaried 
Chiefs of Medicine or Surgery. The radiologist was 
part time and both the pathologist and pharmacist were 
hired consultants. 

Scheduled medical staff meetings included monthly 
business meetings of the entire medical staff, monthly 
meetings of department heads in joint conference with the 
administration and the nursing supervisor, and bi-monthly 
meetings of the Utilization Committee. Besides these 
meetings, no training programs (residency, internship, 
externship) or continuing inedical education programs 
(grand rounds, clinical pathological conferences, x-ray 
conferences) existed at tlospital B prior to PMI con- 
sultation. 

The hospital did not have an adequate library nor 
provisions for meeting place or budgeted funds for de- 
velopment or implementation of an education program. 
Also, the hospital had not established medical education 
prograais in conjunction with other nearby hospitals. 

In August 1967, a PMI trained consultant visited 
Hospital 3 to present PMI's program to help individual 
community hospitaJs set up or improve upon existing con- 
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tinuing medical education programs. The consultant me 
with the Administrator, a staff physician, and the Pres- 
ident of the Hospital Board. Both the Administrator and 
the President of the Board were interested in the Insti- 
tute's program, but wondered where staff doctors, who were 
already very busy, would find time to participate in a new 
program. 

The consultant pointed out that the hospital's reg- 
ularly scheduled clinical and administrative activities 
could include an educational component. He observed 
that in planning an educational program at Hospital B, 
he, in conjunction with PMI , should consider that 1) Hos- 
pital B is a municipal hospital -- which means that all 
approvals of significant expenditures in changes of pro- 
gram must be brought up with a municipal agency, and 
2) the hospital is caught in an ancient rivalry with a neigh- 
boring hospital. 

One week after this initial visit, the con<;ultant 
sent questionnaires to the Administrator for distribution 
to the entire riedical staff in order to obtain data per- 
t^inent to planning a new education program. Ten staff 
physicians cor.pleted the questionnaires . Accordii g to 
a suKkmary of that data, 90 per cent of the staff were willing 
to attend teaching sessions, 90 per cent to instruct, 
100 per cent to be taught by their own hospital staff 
members and SO per cent to use their own patients in teach- 
ing. The staff were willdng to attend, on the average, 
6 hours of scheduled teaching activities per month. 
This summary appeared to indicate substantial staff i iter- 
est in the development of a continuing medical education 
program at Hospital D. 

On Tuesday evening, October 3, the consultant attended 
a dinner meeting i^ith the staff and four members of the 
Board of Trustees, during which he discussed their ideas 
for a continuing medical education program. 

The staff wished to see a r :ating program which in- 
cluded grand rounds , clinical pathological conferences , 
lectures fay staff members or outside speakers and movies 
with staff discussion (omitting the months of June, July, 
August and September). Because of their small number 
and high degree of specialization, the staff believed 
that topics ciiosen for each session should interest the 
hospital staff in general. Ihey also believed that 
they should invite nurses to attend education activities. 
In addition, they decided to invite physicians from a 
neighboring naval base to attend teaching sessions. 

Thp trustees were willing to support the program 
financially. They were prepared to purchase more journals. 
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to help with the preparation of slides, to purchase a 
needed projector and to investigate a proposal that the hos- 
pital allocate a portion of the cafeteria for use as a 
conference area and a lib rary . 

At the end of the meeting, the consultant agreed to 
send Hospital B a proposed program of continuing medical 
education on which the staff and Board o. trustees could 
decide. He also rcrinded the group that an interviewer from 
PMI planned to return in the late winter or early spring 
to follow-up on the hospital's progress. 

As agreed, the consultant sent a proposed program to 
Hospital B at the end of October. His suggested program 
was as f ol 1 ows : 

^. The Board of Trustees and the Administration 
should provide the environmeiiit necessary for 
satisfactory continuing medical education ses- 
sions. These provisions should include the 
following : 

a. A meeting room of adequate size 

The hospital should not use this room ex- 
clusively for education sessions and use of 
the room should not conflict with other hos- 
pital activities . 

b. A room for a small medical library 

This library should include those 
journals selected by a consensus of the staff. 
The hospital should arrange for a naxt-time 
librarian, probably a volunteer from the 
commun ty, to keep these journals in order 
and bound once a year. 

A file of drawers to house records of the 
hospital's education programs 

Perhaps the Ar linistrator could keep a 
log of the day and date of each meeting, the 
topics discussed, the name or unit number 
of the patients reviewed and the staff mem- 
ber in charge of the meeting. Uach chair- 
man should record any special factors which 
help or hinder the educational potential of 
the activity; e.g., new 1 3ons in how to 
plan or put on meeting; conflicts with other 
hospital functions. 
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d. Funds for support of the program 

These funds need not be large, but they 
should be sufficient to include mailing of 
notices, mimeographing of protocols for 
clinical pathological conferences, correspon- 
dence with and expenses for visiting lecturers 
and mailing of invitations. 

2. The Administrator and a staff member (rotated 
on a monthly basis) should share responsibility 
for the education programs at the hospital . After 
some time, perhaps one person can have overall 
cliarge of the program for one year . 

3. In accordance with both written and oral suggestions 
by the staff, the hospital should hold weekly 
education sessions in the daytime, preferably late 
in the morning. These four sessions per month 
(omitting June through September) should include 
the following, on a rotating basis: 

a. A business meeting combined with an edvation 
report f i om a committee - - 

This meeting should consist of a 20 
or 25 minute presentation and/or discussion 
of hospital business important to all the pro- 
fessional staff and a 20 or 25 minute presen- 
tation of a committee report (rotated each 
month) of a hospital activity related to medi- 
cal care, comparing experience at this hos- 
->ital with that of other hospitals as re- 

rted in the literature, evaluating results, 
a discussion of problems. 

b. A specialty round 

This round should rotate in turn among 
each of the specialties represented at the 
hospital. The round should be case-oriented, 
with one or two live or paper cas'^s serving 
as a basis for presentation of a problem from 
that specialty which relates to all the doc- 
tors* experience. For example, the anesthe- 
tist could discuss shock, resuscitation and 
assisted respiration. These topics should 
occur at lea^t once a year for review. 

c. An audit or quality-Cf*re conf erencc.j,-.-and CPC's- 

These eight meetings per year should in- 
clude three clinical pathological conferences 
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in which a staff member discusses a case pro- 
tocol, the radiologist interprets the films, 
the pathol ogist presents the surgical or 
autopsy findings and the staff and the audi- 
ence evaluates the results. Five sessions in 
which staff members review the management of 
specific illnesses should be held such as chole- 
lithiasis, stroke and pneumonia. 

d. Grand Rounds 

Again , staff members should rotate res - 
ponsibility for these rounds each month and 
the rounds should be case-oriented. The phys- 
ician in charge should present one or two 
cases either of in-patients or patients 
asked in for the conference. The discussion 
leader, whether he be the physician in charge 
or another staff member or outside speaker 
should prepare a review of recent literature 
bearing on the case. 

4. The hospital should invite outside speakers ap- 
proxin'ately four times per year. On these occasions, 
the physician in charge that month should send 

the speaker a brief protocol of the case and 

inform him of the size of the audience, the aim 

of the meeting and the nature of the hospital. ^ 

5. The staff should invite the nurses to attend the 
continuing medical education sessions. 

Allowing four months for implementation of a con- 
tinuing medical education program at Hospital B, PMI sent 
a trained interviewer to follow-up the hospital's activities, 
in February 1968. 

During his visit, the interviewer obtained the following 

information: 

- The Board and administration had provided funds for 
continuing education programs; furnished a meeting 
room and space for a medical library; subscribed 

to journals; and provided a desk and ledger for 
meetings, etc. 

- The following education activities were implemented: 

- Specialty rounds were implemented with a frequency 
e xpect ancy of once monthly . 

- Audit conferences (using pathologist and radiologist) 
were established (bi-monthly) 
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- Case Presentatio:. were established (monthly) 

- Lectures were established (twice irionthly ) 
• Grand rounds were es tab li shed (monthly) 

- Nurses were invited to attend education meetings. 

- Educational activities were carefully recorded in- 
dicating date, topic and number of staff and others 
in attendance . 

The consultant visited the hospital again in August 
1968. ^During this visit, he met with the administrator 
and three members of the uedical staff, all of whom ex- 
pressed praise and excitement regarding the medical staff 
education program. Since January ir6S, the hospital had 
conducted a weekly medical staff education activity where 
attendance and e *ui»iasni had been uniformly high. (A copy 
of Medical Staff Education Programs - January 1968 through 
June 1968 may be found on page ) . A preference for 
case presentations and grand rounds had developed, resulting 
in a diminished number of special lectures. 

Responsibility for educational sessions had been ro- 
tated by the staff on a monthly basis. This responsibility 
included supervision of the various education activities, 
specific sessions being divided among other members in in- 
formal rotation. In the month of June, the staff had had a 
strenuous debate concerning the continuation of education 
sessions over the summer. Although some of the staff were 
very enthusiastic, the difficulties of regularly scheduling 
activities during vacation months led to ^discontinuance 
of sessions until September. 

Two alterations in the nursing education program occur- 
red as a spin-off of the physician education program: 
1) staff members began rotating responsibility for presenting 
education material to the nurses; 2) Licensed Practical 
Nurses and other allied health personnel began attending .the 
physicians' programs. Attendance by these individuals at 
physician activities quickly fell off. Responses to a 
quest ionnaire sent by the hospital to memb ers of the LPN 
nursing staff indicated that the physicians' programs were 
going over their heads. As a result, 2 or 3 members of the 
nursing staff assumed the responsibility for conveying per- 
tinent information from the physician sessions to LPN's and 
others at separate afternoon meetings. The head of the 
nursing staff expresses enthusiasm at this beneficial spin- 
off of the physician activity program. 

Finally, the hospital had initiated the sponsorship 
of an outside lecturer from a major medical center; also 
shared by other hospitals within the county's medical soci- 
ety. The activity was well received. 
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The consultant felt that Hospital B had successfully 
implemented a continuing education program and made only 
two additional recommendations which were formally out- 
lined in a letter to the administrator the following 
November: 



1. The hospital should ,-onsider the addition of 4 
outside speakers a year. Sessions should be 
open to neighboring hospitals who in turn might 
share expenses . 

2. Hospital B and neighboring Hospital C should con- 
sider initiation of a *'home and home series 
where each hospital would put on a program utili- 
zing its patients and staff. Also, certain labor- 
atory and radiologic techniques used at one^hos- 
pital and not the other could be demonstrated and 
discussed. At such a time, an outside con suit ant 
could attend to discuss case or disease mechanisms 
pertinent to the experience of both 5 Tfs . 



The following June a PMI trained interviewe visited 
the hospital to check the progress of the consultant's 
recommendations. The hospital was continuing the activities 
developed the previous year, but neither of the new reco- 
mmendations had been implemented although one joint con- 
ference had been held where expenses for a lecturer had 
been shared with Hospital C. 

Organization of a medical library was an additional 
result of PMI consultative efforts at Hospital B In 
October 1968 the hospital entered the Library Program 
offered by PMI (under contract with the National library 
of Medicine), A trainee was sent to attend the Library 
Training Institute held in Boston, February 2-7, 1969, 
sponsored by PMI-NERMLS (New England Regional Medical 
Library Service). The hospital also began acquisition 
of the Medical Core Library developed by PMI for Physi- 
cian Practitioners at community hospitals. 

In total, sixty-two percent of the recommendations 
sent to iiospital B were fully or partially implemented 
ei ghteen months after PMI began its consultation . This 
success was due to the enthusiasm, cooperation and support 
demonstrated by the admini s t r at ion and medical staff. 
The continued rivalry between Hospital B and its neigh- 
boring hospital prevented shared programs of mutual benefit 
and interest, and tliis was the only major failure of the 
education program. 



The administration and staff at Hospital B had im- 
plemented an euucation program through PMI consultation. 
Constructive criticism had been well received and act\on 
to improve the quality of their staff through continuing 
education had been eagerly undertaken. In the words of 
the consultant, ''In general, the program appears to have 
been well received, and to be well launched, and encourage- 
ment along the lines they have already achieved seems to 
me obviously deserved.'' 



j HOSPITAL B 

MEDICAL STAFF EDUCATION PROGRAM - January 4, 1968 thru June 20, 1968 

_ ^ate Topic ^tt.n^-nce 

I'ecture 

Ti ,068 Grand Rounds Staff Members 

J auary 11, ^968 v^j-anu Others 

o o r- JiQ Staff Members 

J nuary 18, 1968 C.P.C. #9 ^^^^^^ 

Purrrf'' Staff Members. 

^ Others 

•1 -iQCQ "Lecture: Cervical Disc 

February 1, 1968 LecT^ure. ^ ^^^^^ Staff Members. 

Others 

R 1968 Grand Rounds Staff Members. 

1 jbruary 8, 1968 vaj-anu Others 

^ n r. JiR Staff Members 

nbruary 15, 1968 C.P.C. #8 Others • 

T«^4-„r.o Staff Members 
February 29, 1968 Lecture 

iQ.o - case presentation: staff Members 

..arch 7, 1968 sarcoidosis. Hypercalcemia Others 

perforated Vistus 

^ ,r R Lecture- Ovarian T-mors Staff Members 

March 14, lSo8 i^ec^uxc Others 

^ iQfio case presentations: St^ff Members 

March 21, 1968 Meningococceminia Others 

Renal Lesion 

'March 28. 1968 lecture - Angiography ^^^^L"!!!!!!. 

■April 4. 1968 • G-„a Rounds ^^^^L"!!*!". 

i^pril 11. 1968 Lecture- «^PalJ--^ ^Sers"!!!!!!. 

Ipril 18. 1968 Ml Female Hormone Day ^3"!-:*!! . 

Kpril 2S. 1968 " Sral^^^^^""' l^^s^^T.'.. 
I 
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Date Topic Attendance 

Muy 2, 1968 Case presentation: Staff Members 11 

pediatric Fracture Review Others 7 

V V Si 1968 Lecture ' Staff Members 8 

Others 8 

h y 16, 1968 Ad Lib Case Presentation Staff Members 7 

Others 4 

1^ y 23, 1968 Guest Lecturer: Staff Members 12 

internal Mammary Others 10 

Transplants 

C ne 6, 1968 Lecture - Anticoagulation Staff Members 7 

Therapy 

0 ne 13, 1968 Case Presentation: Staff Members 7 

Hypertensive (Anemia- Others 6 

Uremia) 

Osteomyelitis - Diabetic 

June 20, 1968 C.P.C. Staff Members 6 

Others 6 



PHASE I 

I 



IMPLEMENTATION (I) AND IMPLEMENTATION PLUS PARTIAL 
IMPLEMENTATION (IPI) RATES OF RECOMMENDATIONS BY 
INDIVIDUAL HOSPITAL 



Hospital 



Number of 
•Reconnendations 



Percent I Percent I 
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APPENDIX K 



CRITERIA FOR CONTENT CLASSIFICATION 
OF PHASE II HOSPITAL CHANGES 
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CRITERIA FOR CONTENT CLASSIFICATION OF HOSPITAL CHANGES 



I. EDUCATION 

A. " Physician education changes include implementation or 

modif icati on of: 

1. medical audits 

2. tumor conferences 

3. clinical pathological conferences 

4. grand rounds 

5. physicians going outside of the hospital for education 
6*. position of Director of Medical Education 

7. PMI or RMP programs for physicians 

B. Nursing education changes include implementation or 
modification of : 

1. in-service education 

2. licensed practical nurse education 

3. nursing audits 

4. nurses going outside of- the hospital for education, i.e., 
ICU-CCU courses 

5. house physicians training nurses 

6. position of Director of In-Service Education 

C. Allied health professions (lab technicians, occupational 
therpasits, physical therapists, dieticians, radiological 
technicians) education changes include implementation or 
modification of : 

1. in-service education 

2. individuals going outside of the hospital for education 

D. Other education changes include implementation or 
modification of: 

1. education programs not specifically designated for any 
target group, e.g., ''now training for ICU-CCU" 

2. education programs for: 



a . 


community 


b. 


nurses aides 


c . 


orderl ies 


d. 


maintenance staff 


e . 


kitchen staff 


f . 


police department 


g- 


fire department 


h. 


pregnant women 


i • 


unwed mothers 


j • 


librarians 



E. 



Facilities and services education changes include 
implementation or modification of: 



1. 
2. 



audio- visual mechanisms 
library 



II. PERSONNEL 



A. 



Phys ici an* personnel changes refer to: 



1. addit iotial physicians (part time, full time, and 
consultants) 

2. replacements of physicians (part time, full'Time, and 
consultants) 

3. other includes: 

a. N . C > (no classification) indicate that there has 
been a physician personnel change, but there was 

no response to the clas:>lfication of either addit ion 
or replacement. 

b. de let ion refers to physician personnel deletion 

due to retirement, death, or elimination of position 

B. Other personnel changes refer to the remainder of the 
hospital staff exclusive of physicians 

1. addit i cnal personnel (part time, full time, and 
cor4Sul':ants) 

2. repilacement of personnel (part time, full time, and 
consul tants) 

3. other includes: 



a. N.C. (no classification) indicate that there has 
been a personnel change, but there was no response 

to the classification of eithf^r addition or replacement 

b. deletion reters to personnel deletion due to the 
el iminat ion of position, death, retirement 



III. EQUIPMENT CHAN'GIiS (exclusive of normal changes in equipment 

and supplies use--: for daily hospital care, e.g., Foley Catheters, 
suture sets, syringes, medications, and linen) 

A. Added equip.uekt includes any recognized change in equipment 
throughout the hospital, e.g., beds, lab equipment, radiological 
equipment, crash carts, and changes to prepared sterile 
supplies , surgery equipment , maintenance equipment . 

B. R eplacement equipment (refer to Equipment A) 

C. N.C. (no c J assi f icat ion) indicates a noted change in equipment 



but not categorized into either added or replacement of equip- 



ment A. 



* 

Note: Each physician change received an individual- classification. 
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IV. PHYSICAL PLANT changes refer to; 

t 

A. New buildings 

B. Raising of buildings 

C. Changing of location of walls 

D. Renovations 

E. New plumbing, sprinkler or electrical installations 

F. New wing 

G. New piped in oxygen which required renovation 

V. SPECIALTY AREA changes. Inclusive for all of the following 
areas are changes concerning the opening or closing of units, 
establishment or elimination of departments, changes in the 
budget, changes in personnel, changes in equipment (additional 
or replacement), changes in education in these areas, change 
in location (wing, floor, or office space), changes in 
procedures . 

A. ICU-CCU changes include any changes related .to equipment, 
services, personnel, etc. of coronary or intensive care units 

B. Radiology changes include changes in radiological diagnostics, 
treatment or therapy 

C. Rehabilitation changes include changes in: 



1. physical therapy 

2. occupational therapy 

D. Lab changes include changes in: 

1. EEG work 

2. EKG work 

3. chemical lab work 

4 . bacteriology 

5 . b lood studies 

6 . urine studies 

7 . pathology 

8 . pulmon ary function Studies 

9. the blood bank 



E. ER (emergency room) changes include changes in: 

1. emergency room personnel 

2. coverage composition and policy 

3. equipment ^or ER 

4 . di sas t er dri 1 1 

F. Mental Health ch anges include changes in: 



1. social work 

2. social service 

3. area of drug abuse and drug .abuse programs 

4. area of alchoholism 
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] G. Inhalation Therap y changes refer specifically to inhalation 

[ therapy, but omit pulmonary functions and pulmonary therapy 

H. Extended Care changes include changes in this area both in 
and ou-tside of the hospital only if the hospital was respon- 
sible for the change. 

I. Administrative Function changes include changes in: 

; 1. budget, with references to: 

a, allocation or reallocation of funds 

b. budgetary categories 

2 . admini stration , with reference to: 

i 

; a. fringebenefits 

b. new department heads 

c. new committees 

I d. inception or elimination of departments 

e. new liaisons 

f. utilization of committee meetings 
I g. trustees 

• h. administrative personnel 

i. outside consultation for the evaluation of facilities 

\ and services 

i j. new personnel policies 

J.' Maintenance Operation changes .include changes in: 

1. buildings and grounds 

2. laundry operation 
j 3. physical plant 

K. OPD changes include changes in: 

i 

1 1. outpatientclinics 

2. OPD services in conjunction with the community 

I L. OR (operating room and Surgery) changes include changes in: 

1. anesthesia 
I 2. recovery room 

VI. COMMUNITY 

A. Changes in educational ^programs for the community, e.g., 
hospital days, pre -nat al programs , drug abuse programs 
I B. Changes in use of VNA. 

VII. NEGATIVE 

Changes relative to elimination of educat ional programs 
Changes where personnel has been dropped from the staff but 
not replaced 

Changes with a loss of an education determinant, e.g., loss 
of educational program, loss of a nursing school 
Changes in decrease of available funds 




dc 



APPENDIX L 



CONTENT ANALYSIS OF EDUCATIONAL AND 
NON-EDUCATIONAL CHANGES-PHASE II 

DETAILED CONTENT ANALYSIS OF 
PHYSICIAN EDUCATION CHANGES- PHASE II 



laoie 



Content Analysis of Educational Changes 



L-la 



Experimental 
Op. Ino. Co» Be. Tot. 



Control 



Op. Ino. CO. Be« Tot. 



E Physician 

R 

S 

Q 

N 

N ^Other 
E/^ Personnel 
L 



Addition 
Replacement 
NO Response 
Other 



1 
1 



Addition 

Replacement 

Other 

NO Response 



11 
2 
1 
4 



11 
2 
1 
9 



6 
2 



9 
1 



9 
2 



i 



J P 
I M 
! E 
|N 



Addition 
Replacenent 
NO Response 



18 2 



2 1 



20 



12 



12 



PHYSICAL PLANT 



S 
P 
E 
C 
I 
A 
L 
T 
Y 

A 
R 

E. 
A 
S 



ICU-CCU 
Radiology 
Rehabil. 
Laboratory 
Emergency Room 
Mental Health 
Inhalation Therapy 
Extended Care 
Budget 

Adm. Fxinction 
Maintenance 
OPD 
OR 



36 
6 
1 
5 
2 
4 
9 
1 
17 
40 



1 
14 



1 

1 1 



1 
2 



I 

42 
9 
1 
7 
3 
4 
10 
il 
20 
53 



52 
•7 
1 
6 
1 
9 
6 

9 
35 



6 
3 

1 
2 

1. 

2 
6 



1 
21 



1 
10 



61 
10 
1 
7 
3 
10 
7 

12 
45 



1 
15 



COMMXJNITY 



36 



43 



25 



30 



KEY/ 



Op B operational (0) 

Ino « initially operational (C Q B) 

Co ■ contemplated (A) 

Be ■ behavioral change 



Table 26a 

Content Analysis of Non-Educational Changes 



L-lb 



22, 



Experimental 
N-42 

Ino. Co. Be, 



Tot 



Control 



N-39 



Op. '• Ino , 



Co, 



Be. Tot 



Physican 



P 

E 
R 

S 

0 

N 

N Other 

E- Personnel 

L 



Addition 
Replacement 
Other 
Not Speci- 



115 53 

18 7 

14 14 

8 3 



3 
4 



Addition 
Replacement 
Other 
Not Speci- 
fied 



87 7 
18 
9 

25 7 



171 


86 


21 


25 


10 


1 ' 


32 


7 


6 


11 


7 


1 


102 


106 


17 


19 


21 


4 


11 


6 




37 


30 


7 



3 
4 



110 
11 
17 
8 



126 
25 

. 6 
39 



E 

Q 
u 
I 
p 

M 
E 
N 
T 



Addition 
Replacement 
Not Specified 



297 


4^* 

0 


S 


347 


266 


35 


2 


303 


20. 


3 


2 


25 


30 


8 


1 


39 


69 


10 


2 • - 


81 


70 


16 


9 


95 


41 


82 


20 


143 


43 


70 


18 


131 



PHYSICAL PLANT 



S 
P 
E 
C 
I 
A 
L 
T 

y 

A 

R 
E 
A 
S 



ICU-CCU 
Radiology 
Rehabil . 
Laboratory 
Emergency Room 
Mental Health 
Inhalation Therapy 
Extended Care 
Budget 

Adm. Function 

Maintenance 

OPD 

• OR • . 



61 
87 
32 
87 
45 
29 
37 
9 
63 
398 
14 
38 
88 



36 
42 
7 
39 
13 
11 
7 
8 
12 
136 

20 
33 



7 
13 
4 
3 
3 
8 
2 
7 
1 
37 

6 
2 



104 

142 
43 

132 
61 
48 
46 
24 
76 

571 
14 
65 
, 123 



47. 26 

73 22 

22 14 

102 39 



52 
36 
33 
4 
SI 
319 
17 
25 
70 



9 
14 

4 

6 
11 
99 

2 
10 
15 



8 
7 
2 
6 
2 



6 
4 



81 
102 
38 
147 
63 
57 
38 
11 
63 
446 
19 
41 
89 



COMMUNITY 



83 



30 13 



126 



64 28 



12 



104 



KEY 

i 

Op s operational (0) 

Ino g initially operat ional^(C_5_B)^ 

Co' contemplated (A) 

Be s behavioral change 



content Analysis of Phase II 



Physician Education Changes Experi- Control 

mental 



A. Resources 




<i86)i 


I 

i 1- 


Fiscal support including hospital, staff and other 
sources 


1191 


\ 7] 


I 


a. 


Establishment of budcet for education 


9 


4 




b. 


Staff monetary contribution or assessment 


1 


1 




c. 


Salary DME 


3 


1 




d. 


Provision of funds for Library 


6 


1 


—J 

2. 


Educational facilities, equipment and materials including 
audio-visual aids and library 








a. 


Provision of Kinescopes, films or proiectors 


20 


19 




b. 


Provision of audio-diqest with tape recorders 


4 


6 


! 


c. 


Use of closed circuit T.V* to outside 


1 


1 


i 


d. 


Use of Albany 2-way radio 


2 




i 


e. 


Use of audio-visual equipment (tape meetings and 
bring back) 


2 


1 


■i 


f. 


Use of training devices (tutors) 


4 


2 


i 

I 


g. 


Development of library including enlargement of 
collection, provision of space & reorganization 
of library 


19 


14 


i 


h. 


Implementation of PMI-NERMLS Core Medical Library 


7 


6 


i 3. 

! 


Hospital Personnel including DME, pathologist, secretaries ^ 
and librarian |19 ' 


Till 


1 


a. 


Establishment of DME, including sharing with another 
hospital (s) 


11 


6 


i 


b. 


Establishment of full or partially paid department 
chiefs or director of medical services 


2 


2 


-J 

i 


c. 


Participation of pathologist and radiologist in 
educational proari-m 


2 


2 




d. 


Provision of librarian - full or part time 


3 


1 


! 


e. 


Provision of secretarial help 


1 






Outside resources including educational agencies v? editing 
consultants and outside speakers 


r4o^ 






a. 


Utilization of PMI 


2 





4 



1 

-1 c. 




2 


Experi- 
mental 


control 




Utilization of 
institutions 


universities or academic 


7 


6 


-"I 


Utilization of 


miscellaneous resources (RHA) 


2 






Utilization of 


outside consultants 


A 

*• 


3 


-]— ^ 

f . 


Utilization of 


outside speakers 


21 


10 


a. ' Planninq & Dcsiqn 









1. program design and assessment including evaluation of 
programs, recommendations, staff needs and interests, 
and comn^unity needs? specification of educational goals? 
structuring of activities and rotating responsibility 
among staff ■ for educational planning and implementation? . 

reorganization of committees lii 

a* Establishment of, or change in education 
cor>mittee & library committee §_ 

b* Assessment of staff needs, interests and 

educational goals . , L 



c* Outline and review of hospital education coals and 
re commend a t ion s ^ ^ ^ 

d* Restructuring or initiation of education program, 
including selection of material & change in number 
and frequency of activities^ 



13 



e* Rotation of staff responsibility for education 
program planning and presentation 



8 



IiiL 



f. Institution of mandatory attendance of M.D.'s & 
Trustees - includinc? review bv-laws 



2* Educational program scheduling and attendance, including 
preparation and publicity of programs, and keeping 



attendance and minutes of meetings 



a* Scheduling of activities on regular basis, well in 
advance # and publicize > 



8 



b> inviting other physicians from community 



1 



Programming 



"TT2T 



1* Patient or case centered acti^'ities including grand 

rounds, CPC's case presentations, and case literature , 

14 31 



review 



a. Establishment of (Grand Rounds) 



12 



b* Establishment of specialty rounds including med*, 
surgical, Ob-Gyn 



1 



c. Establishment of CPC 



d. Establishment of Tumor Clinic 



Experi- Control 
mental 



e. Establishment of case presentation or discussion or 
clinical conferences 



f. Establishment of mortalit y conference 
q. Establishment of x-ray c onference 




3 content centered programs including lecture programs, 
d?2cussion groups and specialty care area education 
programs, e.g., ICU. inhalation therapy 



a . 



Establishment of l *»gturg programs 



b. Establishment of Journal pi scufssion Groups 



4. joint physician-nurse educational activities including 

joint conferences, education programs built around 
specialty care 



5. Miscellaneous training incluuing self assessment test 
nnd po=i>tqrad»»atn training at centers or other hospitals 

6. interhospital Sharing of educational activities and 
teaching personnel ■ 



7. House officer training including resident and intern, 
quest resident, and extern 



b. provision or restructure of intern & Residency 
training program. 



13 



c. 



provision of Medical student or Fxtern program 



8 



1 . Behavior related to Physician Education 

1 




go 



8 1 



8 



8 



a39^\ ^(17 



I 

I 

T 



L-2d 

Experi- Control 
mental . 



2. Alteration of attitudes or outlooks, including 
increased discussion and willingness to teach 



Subtotal A. Resources 

B. Planning and Design 

C. Progranuning 

D. Behavior relrced t-:. / hysician Education 

Total 



137 

41 

122 

39 
339 



1. Alteration of attendance at hospitals and professional •> f- •> 
meetings ! L25j fjQ^ 

ri4l 



86 

17 
82 

17 



202 




APPENDIX M 



"THE IMPACT OF EDUCATIONAL CONSULTATION ON DEVELOP- 
MENT OF CCNTINUING MEDICAL EDUCATION PROGRAMS IN 
COMMUNITY HOSPITALS", NORMAN S. STEARNS, MAR.TORIE 
E. GETCHEIL, ROBERT A. GOLD, AND MARTHA BOUCOUVALAS, 
PRESENT3D AT THE 67TH ANNUAL CONGRESS l J MEDICAL EDUCA- 
TION, SPONSORED BY THE ASSOCIATION FOR HOSPITAL MEDICAL 
EDUCATION, FEBRUARY 12-15, 1971, CHICAGO ,. ILLINOIS . 
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PHASE II ' • 

POSTGRADUATE MEDICAL INSTITUTE 
Evaluation 
HOSPITAL EDUCATION COORDINATOR MEETING 
April 26, 1969 

Name 



Hospital 



I. Please circle the number which best reflects your evaluation of 
April 26th meeting. 

not useful very useful 



II. The most useful part of today's meeting was: (explain) 



III. The least useful part of today's meeting was: (explain^ 



IV. Please cite ar.c! explain one instance of personal learning for you 
today which you think you will attenpt to apply in your capacity 
as consultant or local coordinator. 



What areas or topics would you like to focus on at future licetinRS? 
(Please indicate your choice in order of preference.) 



